Reliable ship detection plays an important role in both military and civil fields. However, it makes the task difficult with high-resolution remote sensing images with complex background and various types of ships with different poses, shapes and scales. Related works mostly used gray and shape features to detect ships, which obtain results with poor robustness and efficiency. To detect ships more automatically and robustly, we propose a novel ship detection method based on the convolutional neural networks (CNNs), called SCNN, fed with specifically designed proposals extracted from the ship model combined with an improved saliency detection method. Firstly we creatively propose two ship models, the “V” ship head model and the “||” ship body one, to localize the ship proposals from the line segments extracted from a test image. Next, for offshore ships with relatively small sizes, which cannot be efficiently picked out by the ship models due to the lack of reliable line segments, we propose an improved saliency detection method to find these proposals. Therefore, these two kinds of ship proposals are fed to the trained CNN for robust and efficient detection. Experimental results on a large amount of representative remote sensing images with different kinds of ships with varied poses, shapes and scales demonstrate the efficiency and robustness of our proposed S-CNN-Based ship detector.