I would like to know the Mutual Information of two variables with no closed-form probability distribution function, from a sample. I am especially interested in the case of continuous variables.
The typical technique here is to construct a joint histogram from your samples, treat the bins as as a finite alphabet and then do the usual calculation. That throws out a lot if information, and is computationally intensive, since it involves sorting your data. That said, a lot of binless techniques are even more computationally expensive, requiring complete k-neighbour graphs or the usual nightmare of kernel density estimation, or fitting of unstable splines, so the binning thing might be an acceptable compromise.
To consider: the relationship between mutual information and 2-dimensional Spatial statistics. And those occasional mentions of calculating mutual information from recurrence plots- how do they work?