All right. Infinite loops, those are no fun at all.

Let's see how things are going towards that goal.

This function call tells it which string to break up.

This is the first step towards our crawler, which will crawl a set of webpages, finding all of the links that can be found from a seed page.

Let me show you a more complicated example. We'll see how it plays out in real life.

Here I've typed it out.

We get the same set of commands now being read from the saved commands variable.

Here's one for HTML, and here's another one for JavaScript.

Let's go and see Pex in action.

So for Question 1, we want to see that you understand how web pages are constructed and what a web crawler will do.

Another super common kind of tag is the anchor tag, which is used to add hyperlinks to webpages.

Their servers are probably on fire, because it's Twitter.

In unit 3, we're going to talk about breaking up tokens and seeing if they match a particular structure, seeing if they're in the language of a formal grammar that describes all of JavaScript or all of web pages.

Let's see some examples.

So typically, programs are very big and accept an infinite set of inputs.

Our web browser will accept as input HTML and JavaScript, the primary languages of the web, and use that to produce an image of the web page.

Let's see whether we get a nice listing together with the covered lines.

So it doesn't look kind of ugly and black and white.

In each item, like in the unordered list is defined with an LI tag.

Otherwise, it would be run away bold.

And now what we want to do is use these regular expressions, together these token definitions to break up a Web page.

You can add a comment containing English text to explain what a program should be doing.

We say Oh, it's something like http.google.com.index.html.

Networks are all about abstraction.

For example, in this case maybe all of that adds up to 6, and that's what we'll display in our resulting webpage.

This is an anchor tag, making a link to Google.

So this should write out 120 and then true.

We'll try a more interesting test.

Google made MapReduce famous building their indices for the web.

This is really nutso. This is not the ideal way of doing things.

And now we've actually got a really good start on our search engine.

People like to surround themselves with unnecessary power, right?

It should look something like this. Here's the front page of a wiki.

What we have built so far is a web crawler.

I'm going to now export this as a style sheet.

The exclamation mark often means factorial in mathematics.

If someone had written this program in JavaScript and put it in the middle of a web page, our web browser would loop forever and never actually render the resulting web page.

That's lexing and parsing, and we do that first.

like this string containing (55).

Another topic that I would like to address is malformed HTML.

If you have a big program like a web browser and you try it on 10 web pages and it seems good, that is not a proof that it's always going to work perfectly in the future, but it does give you more confidence than only trying it on 1 web page.

This is the recorded commands, namely open, save, and quit.

You have to tell us both the type of the tag and also any arguments it might have.

But remember, that HTML can be a collection of elements rather than just 1.

In most cases, at some point the JavaScript code from the user will call write or document_write , and that's that function that says