In a deterministic action, it obviously succeeds, unless of course we run into a wall.

What you have to do with the code we provide is to modify the dynamic programming routine that assumes a deterministic action with the one for a stochastic action.

Let's think about another one.

Let's do another example.

So we're not using this definition anymore.

Those values are discrete.

So is this a discrete or a continuous random variable?

What's the probability of the joint X, Y?

Yes or no.

Question 1 In the first question, I'm going to ask you some very basic probability questions.

They are not discrete values.

And it makes much more sense to talk about the probability or random variable equaling a value, or the probability that it is less than or greater than something or the probability that is has some property

You have discrete random variables, and you have continuous random variables.

We're talking about ones that can take on distinct values.

The motions don't move at all, move right, move down, move down, and move right again.

And I want to think together about whether you would classify them as discrete or continuous random variables.

Y is the mass of a random animal selected at the New Orleans zoo.

The exact, the precise time that you would see at the men's 100 meter dash.

You really don't want to wait for the truck to disappear.

All these problems have in common that they have really challenging planning problems.

So, we actually just learned some interesting lessons.

If there's no cell at all, there's a wall like over here, we assume with 80 chance, we just bounce back to the same cell, but with 10 chance, we instead go left.

What we're going to see in this video is that random variables come in two varieties.

Think about it in terms of the cards.

So the number of ants born tomorrow in the universe.

For those of you that know what a Gaussian random variable is or for those of you that know what a normal random variable is, you can also set W equals Rand N, one by three.

A second terminology for environments pertains to whether the environment is deterministic or stochastic.

One has the outcome that you desire.

If these guys were not here, water would have an equal

The probability of any random variable Y can be written as probability of Y given that some other random variable X assumes value i times probability of X equals i, sums over all possible outcomes i for the (inaudible) variable X.

What's the probability of the complement?

Well let's define one right over here so I'm gonna define random variable capital X and they tend to be denoted by capital letters so random variable capital X, I will define it as is going to be equal to 1 if my fair coin rolls heads and it's going to be equal to 0 if tails

The answer is the measurement, the product, was using Bayes Rule, and motion was using total probability.

You might say OK, that's the probably of getting exactly 1 times the probability of getting 2 out of 3 plus the probability of getting 3 out of 3.

let's look at some actual random variable definitions.

It's 1 if my fair coin is heads.

And if you want it to write, and then you would try to figure it out somehow if you had some information

Give me a chance to change the odds.

It is continuous that is all measurements are continuous and all actions tend to be continuous.

We are again given 5 grid cells.

The is just the definition of conventional probability the joint probability of a set of variables can be factored out as the conditional probability of one variable given all the others, and then we can recursively do that until we've got all the variables accounted for.

Then the value of the state for the action go up would be obtained as follows.

That is, if I execute the east action of this state over here with probability 1 item over here, if I assume the north action over here, probability 1,

The exact precise time could be any value in an interval.

Probability