It would be nice to do the numerical integration via MC estimates using parameters. The problem is that when parameters cross the boundaries of the sub-regions the integrals are discontinuous and the HMC would probably be unhappy.

If you could figure out a way to use parameters, it'd be great because it'd scale to higher dimensions, and you could infer things like multi-dimensional distributions: age, individual income, household income, age of spouse, educational attainment, quantity of meat consumed per year, average duration of commute.

Without a way to calculate how well the distribution fits the observed statistics from a sample of parameters distributed according to the model distribution pdf.... you can't scale up to such multi-dimensional things.

]]>The goal of doing that would be to use the smooth density as a starting point for inferring some other stuff from a causal model of some kind of human or social process.

I think i've come to the conclusion that the "right" way to do this is via Bayesian inference on the parameters that describe the frequency distribution (the GMM for example). Thereby arriving finally at a probability distribution over the frequency distribution.

I'm glad to hear I've convinced you of the kosherness of declarative likelihoods ðŸ˜‰

]]>(Incidentally, the toy problem has an analytical solution: the log-density is piecewise linear with no continuity conditions imposed at the quartile boundaries. The only reason I can see to pick a parametric form like your mixture model is to (in effect) impose a smoothness constraint on the solution. Add enough mixture components and you'll get rapid changes on the quartile boundaries anyway.)

]]>.25 is the observed fraction of people under age 23, and 9 is the observed average age for those people... both of those are data points telling us about a function which happens to have the properties of a pdf, namely f(x) parameterized by our unknown parameters.

so we can put this data on the left, and some function of the parameters on the right of a typical ~ statement in Stan..

.25 ~ normal(integrate(f(x),0,25),SOMETHING)

and

9 ~ normal(integrate(x*f(x),0,25),SOMETHING2)

where SOMETHING and SOMETHING2 are expressions of how big the modeling error and the measurement error and the integration error all put together could be.

Of course, there is no "integrate" function, so instead we need to do some numerical stuff and store the result in a variable, and change our Stan expression appropriately.

We can then do inference on the parameters that control f(x) (namely the GMM parameters) based on this totally legit likelihood.

However, getting it to actually *fit in Stan* is a different story.

----------------------

Update: found a few bugs. I was calculating the averages incorrectly, not dividing by the appropriate normalization constant. Anyway, now it runs pretty well, but if you enforce the constraints very tightly, it has trouble with divergent transitions and slow sampling.

i'll stick the code up and some plots monday.

]]>What about something like

Define a parameterized class of distributions (mixtures for example)

Put very broad priors on the distributional parameters.

Define a set of parameters as independent samples from the distribution above, to be sampled by Stan.

Calculate the estimates of the observed statistics from the sample parameters, and use declarative modeling to create a likelihood that compares the estimates to the observed data.

Ignore entropy.

Do further inference on your model using your sample from your parameterized class of distributions to infer what might have gone on in the detailed data that you only have a table for.

This is essentially Bayesian probability inference over frequency distributions, and it might make more sense in this context.

]]>- set the prior on the parameters to be flat on the appropriate support

- sample N deviates from the mixture distribution for some for large value of N

- compute MC estimates of the constrained quantities and the entropy

- use your declarative modelling strategy to create penalties on the MC estimates in the target

- target += entropy

- optimize

- ???

- profit ]]>