This paper aims at developing a better understanding of the structure of the information that is contained in galaxy surveys, so as to find optimal ways to combine observables from such surveys. We first show how Jaynes’ Maximum Entropy Principle allows us, in the general case, to express the Fisher information content of data sets in terms of the curvature of the Shannon entropy surface with respect to the relevant observables. This allows us to understand the Fisher information content of a data set, once a physical model is specified, independently of the specific way that the data will be processed, and without any assumptions of Gaussianity. This includes as a special case the standard Fisher matrix prescriptions for Gaussian variables widely used in the cosmological community, for instance for power spectra extraction. As an application of this approach, we evaluate the prospects of a joint analysis of weak lensing tracers up to the second order in the shapes distortions, in the case that the noise in each probe can be effectively treated as model-independent. These include the magnification, and the two ellipticity and four flexion fields. At the two-point level, we show that the only effect of treating these observables in combination is a simple scale-dependent decrease in the noise contaminating the accessible spectrum of the lensing E-mode. We provide simple bounds to its extraction by a combination of such probes as well as its quantitative evaluation when the correlations between the noise variables for any two such probes can be ignored.