There are two approaches to analysing data, which I will call the bottom and top half.
Bottom half
Look at images and data by eye. Discover anomalies and outliers. Argue primarily qualitatively. Potentially preprocess the anomaly-finding with machine learning.
It’s a good approach for junior scientists to start looking at data (a) and understand its artifacts and limitations. However, the by-eye approach does not permit quantifying false positives and false negatives.
The bottom half brought us the hunt for high-redshift galaxies and quasars, reddened quasars, the most luminous galaxy, the most massive black hole, Voorwerpjen, etc. etc.
The bottom half is great for discovery of extremes by appearance.
The bottom half approach can only search across data space, not by physical parameters. After discovery, to get physical information for the discovered system, one then switches to another type of analysis.
Top half
Use a physical model to fit data. Build a model that can explain the data. Quantify, including the uncertainties. Distinguish two competing physical processes. This can be achieved with (hierarchical) Bayesian models combined with advanced inference engines (nested sampling and dynamic Hamiltonian Monte Carlo), and more recently, perhaps, with simulation-based inference.
Combining both halves
Some research projects can use only the bottom half, discover something and argue qualitatively. It’s somewhat quite easy to do this: You just project the data into one or two dimensions – for example looking at color – and start visually inspecting the extremes. Slightly fancier versions, but conceptually the same, are to look at a latent space, for example produced by UMAP (or the older T-SNE), or an auto-encoder. These latent spaces are usually meaningless themselves, and randomly oriented so that a retraining gives something completely different. I’ve seen many researchers get lost in over-interpreting the shapes in latent spaces.
To go beyond discovery and qualitative existence proofs, one needs the other half: An approach that models the demographics, placing the extremes in context, and connecting the physical space to the data space with a physically meaningful model.
Admittedly, I call this bottom and top half to make myself and my focus on the top half feel superior. However, the top half can also not exist without the bottom half: You cannot do a blind, automated model fitting without investigating data artifacts, and careful validity checks. You need detailed analyses of individual, nearby galaxies to understand surveys of the distant Universe. So only the two halves make a whole.