Abstract: A detection and localization algorithm was proposed to overcome the issue that unmanned ships has low accuracy in detecting and locating floating objects. The proposed algorithm utilized Faster R-CNN (Faster Regions with Convolutional Neural Network) as the building block to conduct the initial detection and localization, output the temporal result with location boxes. Then the Class Activation network(CA) was used to remove the location boxes and mark the object location with pixels. The case study has verified that the algorithm has satisfactory accuracy in detection and localization of floating objects. Besides, the algorithm remains potential for other objects that shared similar features with lake floating objects.