A policy was not met.

And no matter what policy we have, it's not going to be better than the optimal policy and so we can only be making things worse with our policy, not making them better.

If we tried to go up from this state because that's what the policy said but, stochastically, we slipped over to this state then we wouldn't do something else, according to the policy and so we'd get a little bit of exploration, but we'd only vary from the chosen path because of that variation and we wouldn't intentionally explore enough of the space.

But it learns about the reward function, R, and maybe the transition function, P, if it didn't already know that.

Imagine that you're on a ship in uncharted waters and the captain has a policy for piloting the ship.

So we'll start at a start state, we'll apply the policy and let's say the policy tells us to move in this direction.

Use setting from global policy.

Here's some news for you.

Policies

Policy

In other words, we'd have to stop exploiting the best policy we'd found so far which is this one and start exploring, to see if maybe there's a better policy.

I have mortgage and I have young child and I want to make sure that if anything were to happen to me over the next...

Maybe you know, you live unusually long amount of time, you live fifty years and you're paying or you live to 90s so you live fifty years from the day that you start the policy.

Change Java Policy

Change JavaScript Policy

Overwrite Policy

What does that mean? so that's the difference between the policy that the agent has learned and the optimal policy.

So that policy does, in fact, work randomly making moves with some probability but it tends to be slow to converge.

Attention please!

Your attention, please.

Your attention please.

I'll let them know you're here.

What can be done there?

And Orion is really one of the only, if not the only, off the shelf technologies that could do something. (Laughter)

I've also graphed, with a dotted line, the policy loss.

Java policy

JavaScript policy

So we really have a bound on the utility here.

And finally, we can have a reflex agent where, again, we don't need to know P and R to begin with and we learn directly, the policy, pi of S and then we just go ahead and apply pi.

As you can see, each grid cell now has a label.

I'm closing up the wall over here, opening the wall down here, run it.

I'll alert the Captain.

So we wish to have a planning method that provides an answer no matter where we are and that's called a policy.

He won't.

8 42 exactly.

Default Policy

Duplicate Policy

Default Policy

Send policy

Offline Cache Policy

Realize that even states I'm not likely to every reach,

This is the way of backing up values, and once values have been backed up, this is the way to find the optimal thing to do.

I'll give him the news.

Could be hours, but we'll let you know just as soon as you could see him.

It improved its policy for awhile, but then it got stuck in this policy where we go here, go up and then go right.