Dark matter cannot be observed directly, but its weak gravitational lensing slightly distorts the apparent shapes of background galaxies, making weak lensing one of the most promising probes of cosmology. Several observational studies have measured the effect, and there are currently running and planned efforts to provide even larger and higher-resolution weak lensing maps. Owing to nonlinearities on small scales, the traditional analysis with two-point statistics does not fully capture all of the underlying information5. Multiple inference methods have been proposed to extract more details based on higher-order statistics, peak statistics, Minkowski functionals and recently convolutional neural networks. Here we present an improved convolutional neural network that gives significantly better estimates of the Ωm and σ8 cosmological parameters from simulated weak lensing convergence maps than state-of-art methods and that is also free of systematic bias. We show that the network exploits information in the gradients around peaks, and with this insight we have constructed an easy-to-understand and robust peak-counting algorithm based on the steepness of peaks, instead of their heights. The proposed scheme is even more accurate than the neural network on high-resolution noiseless maps. With shape noise and lower resolution, its relative advantage deteriorates, but it remains more accurate than peak counting.