ICS 32 Final

def connect(host: str, port: int) -> PollingConnection: polling_socket = socket.socket() polling_socket.connect((host, port)) polling_input = polling_socket.makefile('r') polling_output = polling_socket.makefile('w') return PollingConnection( socket = polling_socket, input = polling_input, output = polling_output)

''' Connects to a Polling server running on the given host and listening on the given port, returning a PollingConnection object describing that connection if successful, or raising an exception if the attempt to connect fails. '''

>>> s = 'Boo is happy today' >>> s.upper() #method belonging to str class >>> s.split(' ')

'BOO IS HAPPY TODAY' ['Boo', 'is', 'happy', 'today']

>>> x = Thing() >>> x.a1 = 3 >>> x.a2 = 4 >>> x.a3 = 5 >>> x.a1 + x.a2 + x.a3

12

>>> a = 5 >>> b = 3 >>> a - b

2

HTTP/1.1 200 OK Date: Wed, 31 Jan 2018 07:56:07 GMT Server: Apache/2.2.15 (CentOS) ... ... Content-Length: 435 Content-Type: text/plain; charset=UTF-8 # oops.py # # ICS 32 Winter 2018 # Code Example ... ... if __name__ == '__main__': f()

200 (OK), which means that everything went as planned, the server's way of saying "Okay, cool, here's the web page you asked for!" 404 (Not Found), which means that the server doesn't have the page that you asked to download. The first line of the response is followed by headers, just as the first line of the request is. The server determines what headers to send Date is the date/time at which the response was generated. Server specifies what type of server is being run and what version. As of this writing, the ICS web server is running version 2.2.15 of a server called Apache (which is quite common on the web). Content-Length specifies the length, in bytes, of the content that will be sent back. This allows the client to know when the content has ended. Content-Type specifies what kind of content is being sent back (e.g., a web page, a text file, audio, video, etc.). Browsers respond to the content type by deciding what to do with the content: web pages are shown in the browser, video is often displayed in a video plugin or an external media player, etc. If a browser isn't sure what to do with content, it generally just asks you if you want to save the file somewhere on your hard drive.

Similarly, we can take a string object and call the encode() method on it to turn it back into a bytes object. >>> encoded = text.encode(encoding = 'utf-8') >>> type(encoded)

<class 'bytes'>

If we have a bytes object, we can turn it into the corresponding string by calling the method decode() on it. >>> text = data.decode(encoding = 'utf-8') >>> type(text)

<class 'str'>

Below is the complete Counter class. It consists of four methods: A special initializer method called __init__, which is called whenever an object of this class is being initialized just after it's been created. The count(), peek(), and reset() methods. class Counter: def __init__(self): self._count = 0 def count(self) -> int: self._count += 1 return self._count def peek(self) -> int: return self._count def reset(self) -> None: self._count = 0

>>> c1 = Counter() >>> c1.count() 1 >>> c1.count() 2 >>> c1.count() 3 >>> c1.peek() 3 >>> c2 = Counter() >>> c2.count() 1 >>> c2.peek() 1 >>> c1.peek() 3 >>> c1.count() 4 >>> c1.reset() >>> c1.count() 1 >>> c2.count() 2

The socket module

>>> import socket listening and connecting. Listening means that we want to wait for another program to contact us; connecting means that we want to connect to another program. In this example, since we're writing a client, we'll connect. Once our socket has established a connection — either by receiving a connection when listening, or successfully initiating one when connecting — then we can use it to transfer information between programs, by writing to its output stream and reading from its input stream.

request-response protocol

Client initiates connection to server Server accepts connection Client makes a request Server sends a response

legal clauses

A try and a finally and nothing else A try, at least one except, (optionally) an else, and (optionally) a finally

Importance of storing modules in the same directory

this is the first place that Python looks when you import a module. If there's a module with that name in the same directory, it'll find it. If not, there are a few other places it'll look

Private definitions

Definitions that are only intended to be used as utilities within a module, to allow the writer of that module to break up what might otherwise be large, complex definitions (e.g., a complicated function) into smaller pieces. These make the module easier to write and easier to understand, but do not affect the code in other modules at all.

Public denfinitions

Definitions that you expect other modules to need in order to solve their problems. In other words, these are the tools the module is actually intended to provide to other modules.

def foo(): return 14 def bar(): b1 = 3 * foo() return bi def example(): try: answer = bar() print('The answer is {}'.format(answer)) except: print('ERROR')

ERROR

makefile()

treat a socket similarly to how you treat a text file — reading lines and getting back strings, writing strings — you can do it. Object that behaves outwardly the way a file object does, but that reads and writes via a socket rather than a file.

importance of flush()

File objects — including the ones we get when we call makefile() on a socket — do something called buffering. When you write to them, they store data in memory temporarily and then write it once in a while when the buffer runs out of space. This is mainly done because the act of actually writing the data — especially when you're talking about files on a hard disk, but even when you're using a socket — is much slower than storing it in memory, so only writing data once in a while can dramatically reduce the total cost of a long sequence of small writes. But when you're talking to another program via a socket, it's usually important that a message is sent immediately, because the other program won't know what to do until it is received. For this reason, it becomes important to tell the file object "Take whatever is in your buffer and send it now! Don't wait until an opportune time!" That's what flush() does

urllib.request module has one function that we're interested in

urllib.request.urlopen()

methods in classes

However, we can't ask an integer, a float, a list, or a socket to do the same thing; there is no upper() method in these classes, because it wouldn't make any sense for there to be one. Different classes support different methods that are specific to what each type of object can reasonably do.

>>> connect_address = ('128.195.1.83', 5151) >>> example_socket.connect(connect_address)

If the call to the connect method succeeds, our socket is connected successfully; if the attempt to connect fails, connect will raise an exception instead. The keys to success are that our Internet connection is up and running, and also that there really is a server program running on the host and listening on the port we specified. Once connected successfully, we can use our socket to send and receive data between our client program and the server that we're connected to. The one tricky part is that the data that is sent and received is made up of what are called bytes, which are most directly represented by a Python object with the type bytes. If what we intend to send back and forth is actually text, we would need to perform conversion between the bytes and str types.

repetition

using *for* and *while* loops

conditionality

using *if* statements

moral of recursion

In general, your inclination should be to use loops whenever you can, and recursion when you must.

Sockets

objects that encapsulate and hide many of the underlying details of how a program can connect directly to another program; usually, this connection is made via a network such as the Internet, though it should be pointed out that you can also use ____________ to send data back and forth between programs running on the same machine. provide an abstraction of a connection between a program and some other program.

Data structures

organizing data: the most common four examples of which are lists, tuples, sets, and dictionaries

self

Methods need what seems like an "extra" parameter, usually called self, which represents the object that the method is called on. alternate definition: The reason is that when you make calls to methods, an extra parameter is added before all the others; that parameter is the object the method was called on. By convention, we call that parameter self.

One reason why it's handy for us to see a network connection as two streams

it feels quite a lot like what we're used to doing with files. So far, we've read from files sequentially and written to them sequentially; we'll be able to do the same with sockets. The main difference is that networks are less reliable than files, because so many more things can go wrong in a connection between your computer here in Irvine and one in a faraway place like Korea, so we'll have to be more cognizant of the ways that things can fail; as with most failures in Python programs, these failures will usually arise as exceptions.

>>> from pathlib import Path

Now, anytime you use the word Path, you're specifically asking for the Path type in the pathlib library. Having imported it, you can now create objects of the Path type.

Client initiates a connection POLLING_HELLO boo POLLING_QUESTIONS POLLING_CHOICES 1 POLLING_VOTE 1 1 POLLING_VOTE 1 1 POLLING_GOODBYE closes the connection

Server accepts the connection HELLO QUESTION_COUNT 1 QUESTION 1 Who is your favorite Pekingese? CHOICE_COUNT 1 CHOICE 1 Boo VOTED ALREADY_VOTED GOODBYE closes the connection

>>> s.split(' ') ['Boo', 'is', 'happy', 'today'] >>> str.split(s, ' ') ['Boo', 'is', 'happy', 'today']

This alternative syntax is written by first specifying the name of the class containing the method (in this case, the str class, which is the class from which string objects are built), followed by a dot, followed by the method's name, and followed by its parameters. Notice that the first parameter listed is s, the object on which we wanted to call the method. Whenever we write method calls in the clearer and more familiar form s.split(' '), Python essentially translates these calls behind the scenes to the longer form str.split(s, ' '). While that sounds like a ticky-tack detail that isn't worth knowing — because writing the long-form method calls is generally not a good idea, since it's more verbose and less clear — it's a critical detail if you want to understand how to write classes that contain their own methods, as we'll see. Methods need what seems like an "extra" parameter, usually called self, which represents the object that the method is called on. *Two parameters passed

Polling protocol

like other protocols, governs what each program — a Polling client and the Polling server — is required to send and what it can expect to receive in return. request-reply protocol

enconding

mapping between bytes and their meanings

Two outcomes of a function:

The function will complete its job successfully and return an object of a type you expect. The function will fail to complete its job. Functions fail differently than they succeed in Python; rather than just returning an object that indicates failure, they don't return an object at all, but instead raise an exception.

importance of specifying operations very precisely

The order in which you do things matters, the structure of how you organize your program matters, and, aside from syntax errors, Python will not tell you that your program is wrong in any way other than to give you an output other than the one you expected for a given input.

Why use pathlib and not os.path?

The pathlib library is a relatively recent addition to the Python Standard Library, having been added in version 3.4. Prior to that, there was a library — which still exists, but is a lot less useful — called os.path, which provides functions that manipulate strings instead of Path objects, but still hide some of the details between file systems, such as combining them with the right kinds of slashes depending on what operating system you're using. But using strings and manipulating them with os.path nonetheless leaves you with a lot of room to make mistakes. Strings are simply text; you can store any text you want in a string, whether it's a valid path or not. So you're better off using a tool that was built for the job at hand: when you want to manipulate paths, use the pathlib library.

>>> import urllib.request >>> response = urllib.request.urlopen('http://www.ics.uci.edu/~thornton/ics32a/Notes/Exceptions/oops.py')

The urlopen() function returns an object called an HTTPResponse, which provides a few useful attributes and methods, the most important of which is the read() method, which retrieves all of the content from the response (i.e., the contents of the web page you asked for).

The "loopback" address

There is a special range of IP addresses that can always be used to connect a computer to itself, regardless of what its IP address is. These are called "loopback" addresses, the most common of which is 127.0.0.1. So if you want to test connecting two programs on your own machine, you can use 127.0.0.1 to do that.

Modules should not have code that has observable effect when imported

There should be no global variables (though you can have constants or namedtuples), and the act of importing a module should never cause it to do anything other than make things (e.g., functions, classes, etc.) available that weren't previously.

>>> a = 'Alex' >>> b = 'Boo' >>> a - b

Traceback (most recent call last): File "<pyshell#2>", line 1, in <module> a - b TypeError: unsupported operand type(s) for -: 'str' and 'str'

def f(): x = 3 g(x) def g(n): print(len(n)) if __name__ == '__main__': f()

Traceback (most recent call last): File "C:\Example\oops.py", line 11, in f() File "C:\Example\oops.py", line 3, in f g(x) File "C:\Example\oops.py", line 7, in g print(len(n)) TypeError: object of type 'int' has no len()

urllib.request

We just want to download the contents of a single web page in Python, given its URL

headers

allow us to specify a variety of supplementary information that the server can use to figure out how to send us a response. In our case, we've included just one, a header called Host:, which specifies the name or IP address of the host we think we're connecting to; this is useful in the case that the same machine has multiple names

Importance of Ports

When you want to connect a program to another program running on another machine, it's not enough to know the IP address of the other machine. Multiple programs on the same machine are likely to be connected to the Internet at any given time. So there needs to be a way to identify not only what machine you'd like to connect to, but also which program on that machine you'd like to connect to. The mechanism used for this on the Internet is called a port. A program acting as a server will register its interest in a particular port by binding to it; the operating system will generally only allow one program to be bound to a particular port at a time. Once a program binds to a port on a machine, when a connection is made to that port on that machine, the connection is routed to the program that bound to the port. The important thing to realize here is this: In order for a client to initiate a connection to a server, the client will need to use not only the IP address of the machine where the server is running, but also the port that the server is listening on.

>>> p = Path('D:\\Examples\\data') >>> p >>> type(p)

WindowsPath('D:/Examples/data') <class 'pathlib.WindowsPath'>

>>> p = Path('D:\\Examples\\data') >>> q = p / Path('myfile.txt') >>> q >>> p.exists() >>> p.is_file() >>> p.is_dir() >>> f = q.open('r') >>> f.readlines() >>> f.close() >>> list(p.iterdir()) >>> for x in p.iterdir(): print(x)

WindowsPath('D:/Examples/data/myfile.txt') True False True ['Alex\n', 'is\n', 'happy\n', 'today'] [WindowsPath('D:/Examples/data/myfile.txt'), WindowsPath('D:/Examples/data/test.txt')] D:\Examples\data\myfile.txt D:\Examples\data\test.txt

Tradeoffs when using libraries built by others

You don't get to do everything your way. You sometimes have to adjust your thinking and your design to match what's provided by the library. You may discover that the library doesn't behave the way it's been documented, that the documentation is inadequate and doesn't tell you what you need to know, or that the code is simply buggy in places. You may find that there are vital things you need that the library doesn't provide, even if it provides some of what you need.

Classes

a blueprint for a new kind of object. That blueprint specifies what objects of the class can do, by defining a set of methods, each representing one operation these objects are capable of performing. For example, the str class, which is built into Python, contains a number of methods we've seen, like upper(), split(), and startswith(). We define a new class by using the class construct in Python. The simplest kind of class is one that doesn't know how to do much of anything (aside from a few built-in things that all objects can do), which we could write this way. class Thing: pass As in if statements and loops, pass is used to specify that a class is empty. An empty class isn't a very interesting one, and we'll want to write ones that do more. First, though, we need to know a little bit about the mechanics of method calls in Python, which aren't quite as they seem, and our Thing class will help us to learn those mechanics.

Display of IP Addresses

a sequence of four numbers separated by dots; each of the numbers has a value in the range 0-255 (a range chosen because values in this range can be stored in eight bits, or one byte). For example, as of this writing, the IP address of one of the machines that act as a "server" for the ICS web site has the address 128.195.1.76.

parameters

accepted by functions to control how they behave

The Domain Name System and DNS lookups

acts as a kind of phone book; given the name of a computer, ____ can tell you its IP address, so long as that name has been registered with ____ previously. unlikely to affect your connections to programs on other machines significantly in this course, other than the fact that you may want to use a hostname instead of an IP address when you know one

IP addresses

akin to a telephone number for every machine connected to the internet network will be able to determine who should receive the message and how the message should get there, hiding these details from the machines on either end. many machines have a different IP address as often as every time they connect to the Internet

Importance of abstraction

allows you to write programs that are 50,000 lines of code instead of just 50, to write programs that take months or years to complete instead of just hours or days, and to write programs together with ten other people instead of only working by yourself. allows you to effectively use code written by others, even if you've never looked at it and wouldn't necessarily understand it if you did. In all of these cases, the primary benefit is that no one needs to carry every detail of the entire program in their heads at any one time; you can do that with a 50-line program you wrote yourself, but not with a 50,000 line program you wrote with ten other people. -tl;dr the only reason we've been able to build the incredible collection of hardware and software that runs so many things in our world.

>>> data = response.read() >>> response.close() >>> data

b"# oops.py\r\n#\r\n# ICS 32 Winter 2018\r\n# Code Example\r\n#\r\n......."

Dangers of except clause with no type listed

because they'll handle every kind of problem the same way, meaning even a program bug (like misspelling the name of a variable) will be handled the same way as the kinds of problems you expect (like a file not existing when your program tries to open it).

Importing modules

causes the code in that module to execute — so, for example, if it contains five function definitions, those functions are now defined — and then makes its definitions available to the module that imported it.

GET

client would like to "get" a resource (a web page, an image, etc.) from the server. (We may see other alternatives later if we find a need for them.) A GET request in HTTP/1.1 looks like this. GET /~thornton/ics32a/Notes/Exceptions/oops.py HTTP/1.1 Host: www.ics.uci.edu

bytes object

collection of bytes

Software libraries

collection of functions that are already implemented to solve a particular kind of problem. It's a set of operations (and maybe even some information). Ideally, they also include documentation that explains how to use them, e.g., what the names of the functions are, what inputs they accept, what outputs and effects they give as a result, and in what ways they might fail.

network address translation (NAT)

converts the internal, "fake" IP addresses used by my computers to its own IP address for traffic going out, and converts its own IP address back to the "fake" IP addresses on the way back in.

dictionary-shaped

data identified by unique keys, like student information with a student ID number associated with each one

information

data that your program will work with. What are its inputs? What intermediate results will it remember while the program runs? What are its outputs?

Example of recursion adding up the integers in a nested list of integers

def nested_sum(nested_list: 'nested list of integers') -> int: '''Adds up the integers in a nested list of integers''' total = 0 for element in nested_list: if type(element) == list: total += nested_sum(element) else: total += element return total

namespace

dictionary that maps names to the objects that have those names

from x import *

dumps a potentially large set of names into a module's global namespace, colliding with any same-named values in my own module. And every time the x module changes, it might introduce a new conflict that wasn't there before. This is a dangerous choice, especially in a program that will have a long life, as so many real-world programs do

Coupling

extent to which one module's code relies on the details of others; low coupling means that each module relies on individual details of the others as little as possible. As a practical matter, of course, there will need to be at least some coupling between modules (e.g., if one module needs to call a function in another), but a good rule of thumb is that each module should reveal as few of its details to the other modules as possible.

Router

forward outgoing traffic from each computer to the single Internet connection, and to take the incoming traffic and route it to the appropriate computer.

Ports

have numbers that range from 0-65535 (a range chosen because values in this range can be stored in sixteen bits, or two bytes). It's generally a good idea not to use ports with numbers below 1024, because these tend to be reserved for common uses (e.g., web traffic, FTP traffic, email traffic).

Objects

have types, and their types determine what operations can be performed on them

The definitions in your modules should have what software engineers call

high cohesion and low coupling

recursive

included in its own definition

Way to separate public definitions from private definitions

prefixing the names of "private" definitions with a single underscore ('_') character. Since the private functions, by their nature, are the ones that are more likely to be changed, added, or removed over time, the underscore serves as a sort of warning to users of the module that they should exercise caution and quite probably stay away from such functions, because they're likely to change in ways that will break callers.

client

program that initiated the contact

HTTP (HyperText Transfer Protocol)

protocol with which most web traffic on the Internet is transacted. Its latest version is HTTP/2.0, though it's still in the early stages of worldwide adoption; we'll stick with the more broadly-used (and easily-understood) HTTP/1.1 for now.

Importance of reading a traceback

provides a lot of useful information, even if you don't quite understand the error message at first, because they don't only tell us what the error was, but also where the error occurred.

>>> input_stream = example_socket.makefile('r') >>> output_stream = example_socket.makefile('w')

reading and writing of a socket

input stream

receives all of the data sent by the other program, in the order the other program sent it.

Cohesion

refers to the extent to which the definitions in a module are related to one another; high cohesion means that they're highly related. In short, do they "belong together"? Functions that are all involved in implementing a communication protocol probably belong together; functions that are involved in implementing a protocol probably don't belong together with functions that comprise part of a user interface or print error messages in the Python interpreter.

Path objects

represent paths on a file system. They aren't files and they aren't strings; they're paths, which means they are explicitly intended to represent the way that a file system keeps track of where a file is. Since there are substantial commonalities between different kinds of file systems, there can be one kind of object that represents those commonalities.

server

responds to the initiation

result

returned by a function. might be the built-in value None if it's a function that is primarily intended to provide kind of side effect (e.g., printing output, writing to a file).

local-area network (LAN)

router assigns a "fake" IP address to each of my computers, using a range of addresses that is never assigned to computers on the Internet

list-shaped

sense that you have a sequence of values where the order of the elements in the sequence is considered relevant

>>> example_socket = socket.socket()

socket object

localhost

special name for "loopback" address

Protocols

specifies what each program will send and what it will expect to receive. As with file formats, there are well-known protocols already defined for specific purposes — like the HTTP protocol that describes how data is transferred over the web, or the SMTP protocol that is used to send email — but you can also define your own protocol if you need something specific for your particular use. What's important is that both programs implement the same protocol, and that each program knows its role in that protocol.

finally:

statements that will always execute after leaving the "try", whether an exception was raised or not note that these statements will happen after any in the "except" or "else" that also need to execute primarily used for what you might call cleanup

try:

statements that will be attempted once if any exception is raised, control leaves the "try" clause immediately

except:

statements that will execute after any statement in the "try" clause raises an exception can optionally — and, more often than not, they do — specify a type of exception that they handle

else:

statements that will execute after leaving the "try", but only if no exception was raised

output stream

takes any data written to it and sends it to the other program, in the order it was written.

Abstraction

taking complexity and hiding it beneath a veil of simplicity, allowing you to focus on the details of how something is used, while being able to ignore the details of how it's built inside.

Executable modules (if __name__ == '__main__')

we can simply check the module's name by accessing the __name__ variable. If its value is '__main__', the module has been executed; if not, the module has been imported. So, in an executable module, we typically write the code that causes things to happen when the module is executed in an if __name__ == '__main__': block, so that it will only happen if the module has been executed. Meanwhile, if the module is imported, its definitions will become available to another module, but there will otherwise be no effect.

operation

what your program does, which you'll generally have specified by writing your own Python functions, or by calling functions that are built into Python already.

>>> input_stream.close() >>> output_stream.close() >>> s.close()

when we're done with the socket, we'll want to close both the pseudo-file objects and the socket itself, to let the other program know that we've finished our conversation, and also to release any operating system resources associated with the open connection. The right order is to first close the two pseudo-file objects, then to close the socket.

Firewalls

which are software or hardware that restrict other computers' access to computers behind them. Internet service provides have their own firewalls and traffic limitations in place, so if you're working from home, your experience — especially in terms of being able to have others connect to you — may vary considerably depending on your provider.

Control structures

which govern how your program flows from one part to the next

Functions

which let you organize your program into small, relatively independent units of work

importance of '\r\n'

without which the receiving program won't know that the sender has sent a complete line of text.

If you want to use a library:

you have to understand something about the problem domain — the kind of problem that it solves, the terminology that's used to describe aspects of that kind of problem, and the basic concepts involved in its solution. You have to be able to understand the contract between you and those libraries: what inputs you need to provide and what outputs and effects you can expect in return. Even if there's well-written documentation, you'll still have to spend time reading it, understanding it, and determining whether the library really is a good fit for solving some part of the problem you have.

'\r\n'

— technically, these characters are known as a carriage return and a line feed