41–50 of 56 posts
Earlier quoted context omitted.
you are assuming that the reader is familiar with this specific understanding of "derive". someone not familiar with formal methods wouldn't.
what other understanding of derive is there???
Derivative financial instruments - e.g. options, CDSs.
I'm sure there are many more examples.
Earlier quoted context omitted.
Let's talk about a fair dice roll to make it concrete, and let the rolled number be X and let the event that we rolled an even number be E. P(X=6|E) = 1/3. P(X|E) is a distribution where 1,3,5 has 0 probability mass and 2,4,6 have 1/3 each. If we consider X|E as a random variable, what is its value if we roll an odd number? Undefined? What does that mean? Random variables always have some value. Sure you can build a…
> If we consider X|E as a random variable, what is its value if we roll an odd number? Undefined? What does that mean? Random variables always have some value. Random variables have some value on their domain, and for the random variable X | E=1 the sample space is restricted to the elementary events {2,4,6} which conform the composite event E=1. The original sample space is partitioned in the subspaces {1,3,5} and {…
Of course it doesn't! You first have to define them on a common space (the Cartesian product), and for that you have to specify their joint probabilities. One example might be that you model them as independent. Otherwise we wouldn't know how the coin and the dice relate. Sure independence is usually a good default assumption, but it's still a necessary step.
Earlier quoted context omitted.
> If we consider X|E as a random variable, what is its value if we roll an odd number? Undefined? What does that mean? Random variables always have some value. Random variables have some value on their domain, and for the random variable X | E=1 the sample space is restricted to the elementary events {2,4,6} which conform the composite event E=1. The original sample space is partitioned in the subspaces {1,3,5} and {…
> You could do the same for a roll of a dice and the toss of a coin. Or do you think that computing the joint distribution of a coin toss and a dice roll doesn't make sense because they are defined over different spaces? Of course it doesn't! You first have to define them on a common space (the Cartesian product), and for that you have to specify their joint probabilities. One example might be that you model them as…
> Let's consider two independent rolls, X and Y. You can't compute the joint distribution P(Y, (X|E)), it just doesn't make sense as the two "variables" are defined over different spaces.
Do you agree that you cannot compute the joint distribution P(Y,X) either because the two variables are defined over different spaces?
Earlier quoted context omitted.
Let's talk about a fair dice roll to make it concrete, and let the rolled number be X and let the event that we rolled an even number be E. P(X=6|E) = 1/3. P(X|E) is a distribution where 1,3,5 has 0 probability mass and 2,4,6 have 1/3 each. If we consider X|E as a random variable, what is its value if we roll an odd number? Undefined? What does that mean? Random variables always have some value. Sure you can build a…
Sure you can. The TLDR would be "piecewise constant projection" I think picking up a standard graduate probability book will clear this up better than any long comment trail. There are no problems defining a coarser sigma algebra using an original one and then defining a function measurable on the new sigma algebra. Note this continues to be an r.v. in the original space as meaurability is preserved. A consistent def…
Can you recommend one? I just picked up Probability and Measure by Billingsley and it does not mention "conditional random variable" a single time in over 600 pages. It does have a lot of "conditional probability", "conditional distribution", "conditional expectation" etc.
> You seem to be accepting of a conditional expectation.
Conditional expectation is defined in terms of conditional probabilities, and those are in turn explicitly defined as P(A|B)=P(A,B)/P(B), so there's nothing not to accept.
Earlier quoted context omitted.
Sure you can. The TLDR would be "piecewise constant projection" I think picking up a standard graduate probability book will clear this up better than any long comment trail. There are no problems defining a coarser sigma algebra using an original one and then defining a function measurable on the new sigma algebra. Note this continues to be an r.v. in the original space as meaurability is preserved. A consistent def…
> I think picking up a standard graduate probability book will clear this up better than any long comment trail. Can you recommend one? I just picked up Probability and Measure by Billingsley and it does not mention "conditional random variable" a single time in over 600 pages. It does have a lot of "conditional probability", "conditional distribution", "conditional expectation" etc. > You seem to be accepting of a c…
Any book that explains conditioning over a sigma algebra should suffice. You could try Loeve, Dudely or Neveu but dont remember if its mentioned explicitly.
BTW conditional expectation is really more fundamental than conditional probability. Its the former that yields the latter in measure theoretic probability. If you want to drink from the source that would be Kolmogorov.
Finally if you are reading Billingsley you are adequately qualified to call yourself a mathematician.
In my very ignorant world view Kalman filters make some prediction from some input. As do various ML techniques. As do various statistical models and techniques. What are some good sources that show the connection between all these techniques and help me pick the right one for specific use cases? For example, I want to predict a time series let's say the number of visitors of a site. I know some characteristics of th…
The word "filter" indicates that it turns one sequence (usually a time series of measurements) into another sequence (an estimate for underlying states). You can think of it as denoising the sequence of measurement by using knowledge about how the underlying system behaves. For example, if our GPS measurement says we suddenly jumped 100 meters compared to a second ago, we can weigh this against our prior knowledge that a car (the underlying system) is not likely to make such a sudden position change.
The Kalman filter weights the incoming measurement against what our model would predict. Both the prediction and the measurement are probabilistic and they are weighted according to the uncertainties of them. The more certain source of information is weighted higher.
> For example, I want to predict a time series let's say the number of visitors of a site. I know some characteristics of the series (periodic, seasonal), but how should I go about it?
That's not what the Kalman filter is for as you are not trying to denoise some sequence of noisy measurements.
Earlier quoted context omitted.
> You could do the same for a roll of a dice and the toss of a coin. Or do you think that computing the joint distribution of a coin toss and a dice roll doesn't make sense because they are defined over different spaces? Of course it doesn't! You first have to define them on a common space (the Cartesian product), and for that you have to specify their joint probabilities. One example might be that you model them as…
What did you mean with the following paragraph then? > Let's consider two independent rolls, X and Y. You can't compute the joint distribution P(Y, (X|E)), it just doesn't make sense as the two "variables" are defined over different spaces. Do you agree that you cannot compute the joint distribution P(Y,X) either because the two variables are defined over different spaces?
Earlier quoted context omitted.
What did you mean with the following paragraph then? > Let's consider two independent rolls, X and Y. You can't compute the joint distribution P(Y, (X|E)), it just doesn't make sense as the two "variables" are defined over different spaces. Do you agree that you cannot compute the joint distribution P(Y,X) either because the two variables are defined over different spaces?
I meant them to be defined on the same space. It's a single experiment, the outcome of which are two rolls that happen to be independent.
But the fact that each variable alone is defined on the "same" sample space {1,2,3,4,5,6} is irrelevant.
The situation is no different from the joint probability for random variables X and Z corresponding to a single experiment consisting of a dice roll and a coin toss, where the relevant space is the cartesian product of {x=1,x=2,x=3,x=4,x=5,x=6} and {z=1,z=2}.
And it is also similar for the situation you asked about, with a random variable Y and a "conditional" random variable X|Even. The relevant space is the cartesian product of {y=1,y=2,y=3,y=4,y=5,y=6} and {x=2,x=4,x=6}.
Earlier quoted context omitted.
I meant them to be defined on the same space. It's a single experiment, the outcome of which are two rolls that happen to be independent.
If you mean that the space for this single experiment composed of two rolls (random variables X and Y) is the cartesian product of {x=1,x=2,x=3,x=4,x=5,x=6} and {y=1,y=2,y=3,y=4,y=5,y=6}, then I agree. But the fact that each variable alone is defined on the "same" sample space {1,2,3,4,5,6} is irrelevant. The situation is no different from the joint probability for random variables X and Z corresponding to a single e…
Let's say T2|O=1 is a "conditional random variable". Let's consider the average temperature indoors and outdoors. What would ((T1|O=1) + T2)/2 even mean? How could you use the two "variables" in the same expression? What is even their joint distribution? They are defined over different spaces!
This means, we must always carefully condition all variables used together on the exact same things. So ((T1|O=1) + (T2|O=1))/2 is valid. But then why do this on all variable instances that we use? It would be very tedious. At some point we want to get to a distribution (or some function of a distribution, like the expectation or variance), so it's much simpler to say for example P((T1 + T2)/2 | O=1), which is just a good old conditional distribution. Conditioning is an operation on a distribution and in my mind the bar (|) is really a slot in the P() notation and is short for P(A,B)/P(B). A bar popping up elsewhere (like in expectations) must be directly determined by the distribution (a random variable is not).
Overall, since you cannot mix differently conditioned "conditional random variables" in a single expression, you may just as well put your conditioning on the side of the whole expression in the P().
Not sure if my point is coming across...
Earlier quoted context omitted.
> I think picking up a standard graduate probability book will clear this up better than any long comment trail. Can you recommend one? I just picked up Probability and Measure by Billingsley and it does not mention "conditional random variable" a single time in over 600 pages. It does have a lot of "conditional probability", "conditional distribution", "conditional expectation" etc. > You seem to be accepting of a c…
Billingsley is pretty darn good. It might have left the connection as a dotted line given that the notion is no different from conditional expectation. The only connection you have to make is conditional expectation is a function and a random variable. You must have seen expectation taken of a conditional expectation. That should should convince you that condititional expectation is indeed a random variable. Since th…
While I agree with isolated points of your comment I think it doesn't add up to a useful/coherent concept of conditional random variable.