In essentially all applied sciences, data-driven modeling heavily relies on a sound calibration of model parameters to measured data for making probabilistic predictions. Bayesian statistics is a consistent framework for parameter inference where knowledge about model parameters is expressed through probability distributions and updated using measured data. However, Bayesian inference with non-trivial stochastic models can become computationally extremely expensive and it is therefore hardly ever applied. In recent years, sophisticated and scalable algorithms have emerged, which have the potential of making Bayesian inference for complex stochastic models feasible, even for very large data sets. We investigate the power of both Approximate Bayesian Computation (ABC) and Hamiltonian Monte Carlo (HMC) algorithms through a case study in SOLAR PHYSICS. Time-series of cosmogenic radionuclides in wood and polar ice cores are a proxy for solar magnetic activity on multi-millennial time-scales and exhibit a number of interesting and mostly not-yet-understood features such as stable cycles, Grand Minima and intermittency. Solar physicists have put a lot of effort into the development of stochastic solar dynamo models, which need to be calibrated to the observations. Parameter inference for stochastic dynamo models on long time-series of radionuclides is an open and highly topical question in solar physics. Achieving more reliable predictions of solar activity has important implications in environmental and life sciences.