CS409 Part 3

Architectural Style: Layered Systems

"A layered system is organised hierarchically, each layer providing service to the layer above it and serving as a client to the layer below." (Garlan and Shaw) AKA Virtual Machines. • Layers may be hidden to all except adjacent layers or layers may be partially hidden.

Pipes and Filters (Pipeline) Architectural Style II

"The Pipes and Filters architectural pattern [style] provides a structure for systems that process a stream of data. Each processing step is encapsulated in a filter component. Data is passed through pipes between adjacent filters. Recombining filters allows you to build families of related systems." [POSA96] p53

KWIC Pipe and Filter

4 Filter components: Input, Shift, Alphabetize, Output. • Each filter (incrementally) process data and passes to next filter. • Control distributed - each filter can run when it has necessary data. • Data sharing strictly limited to that transmitted on Pipes

Event Driven Implicit Invocation II

A collection of loosely coupled objects connected by events. As you'd expect using Observer. No hardwiring of method calls - can attach and detach as necessary to modify behaviour. • Strength: relatively easy to add, remove and replace objects / modules e.g. an object that displays words containing 'z' or over a certain length. • Weakness: can be very difficult to understand what is happening at runtime. Especially if overused in large system. Tracking down bugs difficult. • Design challenge: identify appropriate classes, events and dependencies. • Commonly used style in creating flexible frameworks. Plug-in new modules that react to events and broadcast own events. • In general mix implicit invocation with conventional explicit invocation e.g. stop_word_filter.is_stop_word(w)

code golf

Aim is code brevity i.e. score as low as you can ... • Use of advanced features and libraries e.g. collections.Counter, counts.most_common, re.findall(...) • Can be elegant but usually more difficult to understand and error prone. • Clarity always beats brevity ... [ Some functional differences in these programs ... treatment of alphanumerics and underscores ]

Case Study: Key Word In Context

Aims: - To demonstrate key features of four architectural styles. - To identify relative strengths and weaknesses of these four architectural styles. • First proposed by David Parnas as an example to demonstrate information hiding - key idea behind OO. The problem: "The KWIC system index system accepts an ordered set of lines, each line is an ordered set of words, and each word is an ordered set of characters. Any line may be "circularly shifted" by repeatedly removing the first word and appending it at the end of the line. The KWIC index system outputs a listing of all circular shifts of all lines in alphabetical order." [Garlan and Shaw paper pp16-22] • Widely used in Computer Science: - Unix man page permutated index - Keyword in context indexes for libraries

Software Architectural styles

Aims: - To explain the role of software architecture in software engineering development process. - To explain and demonstrate key architectural styles/patterns » Pipes and Filters » Object-Oriented (Data Abstraction - ADT-based) » Subroutine and Shared Data » Event-Based, Implicit Invocation » Layered - To use well known case-study to demonstrate, compare and contrast architectural styles Keyword in Context

Layered Systems

Applicability: - a large system that is characterised by a mix of high and low level issues, where high level issues depend on lower level ones. • Components: - group of subtasks which implement a 'virtual machine' at some layer in the hierarchy • Connectors: - protocols / interface that define how the layers will interact • Invariants: - limit layer (component) interactions to adjacent layers (in practice this may be relaxed for efficiency reasons)

Pipe and Filter: weaknesses

Can degenerate to 'batch processing' - filter processes all of its data before passing on (rather than incrementally). • Sharing global data is expensive or limiting. • Can be difficult to design incremental filters. • Not appropriate for interactive applications - they don't split into sequential stages. POSA book has specific styles for interactive systems, one of which is Model-View-Controller. • Error handling is Achilles heel e.g. filter has consumed threequarters of its input and produced half its output and some intermediate filter crashes! Generally restart pipeline. [POSA] • Implementation may force lowest common denominator on data transmission e.g. Unix scripts everything is ASCII.

Comparison Criteria

Change in overall processing algorithm: line shifting can be performed as line read in, on all lines after they are read, or on demand when sorting requires a new set of shifted lines. • Change in data representation: circular shifts can be stored explicitly or implicitly as indices into the original lines. Different data structures can be used. • Enhancements: eliminate shifts that start with noise words ("a", "the"), allow deletion of lines, make system interactive • Performance: space and time. • Reuse : to what extent may components be reused? Four well known, published and implemented solutions based on different architectural styles

Implicit Invocation Example (Architectural)

Common style for integrating tools in a shared environment. • Garlan and Shaw mention the Field tool integration environment: - Tools communicate by broadcasting interesting events - Other tools register patterns that indicate which events should be routed to them and which method/procedure should be invoked when an event matches that pattern. - Pattern matcher responsible for invoking appropriate methods when each event is announced. • Examples: - Editor announces it has finished editing a module, compiler registers for such announcements and automatically re-compiles module. - Debugger announces it has reached a breakpoint, editor registers interest in such announcements and automatically scrolls to relevant source line.

Procedural

Constraints • Abstractions are procedures ('subroutines' or methods in Java) • Procedures share global data • Problem solved by procedural decomposition - break problem down into sequence of subproblems, recursively break subproblems down until solved by small procedure. Each procedure 'does one thing' / performs one 'well-defined' task. Procedures cause state changes (side-effects) on global data Main program is a sequence of procedure calls solving top-level problem Can pass more data in and out as parameters

Object Oriented II

Constraints • Objects encapsulate data exposed through controlled interface (methods / functions). • Data never accessed directly. • Objects can be replaced by other objects with different implementations so long as the interface is the same - polymorphism. • This style often associated with classes and inheritance - though neither necessary nor sufficient for object oriented style.

Event Driven Implicit Invocation

Constraints • Problem decomposed into entities usually using objects (could be nonOO modules or procedures). • The entities are never called directly for actions - they are called implicitly (as opposed to 'explicit' method calls where callee is known) • Entities provide interfaces for other entities to register for 'callbacks'. • At certain points in computation, entities call on the other entities that have registered for callbacks. Similar to using Observer Design Pattern for all entity communication. AKA Hollywood - 'Don't call us ... we'll call you'

Pipeline (pipe and filter)

Constraints • Problem decomposed using functional abstraction. • Functions perform one well defined task - take input and produce output. • No shared state between functions. • In pure form, nothing exists outside function other than source of data input to function and destination for output. • In pure form all data passed in/out of every function is same type e.g. sequences of characters. • Tend to be easier to test (no state) and to distribute concurrently. • Well suited for problems that are decomposed into sequence of processes acting on stream(s) of data. • Not all problems can be decomposed like this ...

Layered System Examples

Example 1: ISO defined the OSI 7-layer architectural model with layers: Application, Presentation, ..., Data, Physical. Protocol specifies behaviour at each level of abstraction (layer). Each layer deals with specific level of communication and uses services of the next lower level. • Example 2: TCP/IP is the basic communications protocol used on the internet. POSA book describes 4 (5 now!) layers: ftp, tcp, ip, Ethernet. The same layers in a network communicate 'virtually'. • Example 3: Operating systems e.g. hardware layer, ..., kernel, resource management, ... user level. "Onion Skin model". • Example 4: Java Virtual Machine - platform independent layer to provide portability.

KWIC Implicit Invocation

Implicit invocation - Observer-like notification of change. • Components similar to Abstract Data Type. Abstract interface hides data representation. But: • Components invoked implicitly when data modified - active data model. • Adding a new line to Lines causes an event to be sent to Circular Shift module which puts a circular shift of the line in separate, abstract, shared-data store. • The Alphabetizer is triggered by the completion of shifter activities to sort the lines in the Shifted Lines buffer. Can be based on shared or separate buffers. The sort can be incremental (e.g. by insertion) or on the complete buffer. • Alphabetizer could trigger Output.

Conclusions (architectural styles)

Important to be aware that there are practical styles other than object-oriented. • Large scale practical software systems often combine a variety of architectural styles. Particularly, web-based systems: database, client-server, object-oriented components, ... • Software architecture styles/patterns are relatively new and still evolving - though there is consistency from [SG96] to [TMD09]. • As with Design Patterns, Architectural and Programming Styles provide a more sophisticated vocabulary for discussing design alternatives.

KWIC Example

Input: Pattern-Oriented Software Architecture Software Architecture Introducing Design Patterns Output (assuming Pattern-Oriented treated as one word): Architecture Software Architecture Pattern-Oriented Software Design Patterns Introducing Introducing Design Patterns Patterns Introducing Design Pattern-Oriented Software Architecture Software Architecture Software Architecture Pattern-Oriented • Can now quickly search for titles that contain phrases such as "Software Architecture" or "Design Pattern" ...

The Problem: Term Frequency

Inputs: A text file containing any series of words e.g. text of a book Stop words file, list of basic words to ignore e.g. "a", "the", "is" ... Output: The number of times each non-stop word occurs in input text file Ordered from highest occurring to lowest, limited to n=25, top occurring words. All 33 programs are written in Python and available on GitHub: http://github.com/crista/exercises-in-programming-style

Programming style

Next two lectures look at some examples of Programming Style. • Key idea is the programming constraints imposed by the style. • Strong relationship with 'Architectural Style' (lectures 3 and 4) but demonstrated at level of programs. • Similar styles ... but slightly different names. • Architectural style imposes constraints ('invariants') at architectural level, programming style imposes similar constraints at program level. • Strengths and weaknesses follow from the constraints. • You should be able to recognise, understand and use a range of styles both in design and programming.

KWIC Abstract Data Type (OO)

Similar set of modules to Main Program / Subroutine. • Control algorithm is similar. • Key Difference: Data not directly shared - data accessed only through module interfaces. • Circular Shift and Alphabetic Shifts typically hide shifted/sorted copies of the original lines (although Parnas' 1972 solution avoided this).

Software Architecture

Software architecture is the software engineering activity concerned with designing and specifying the overall system structure. • It is concerned with "principle design decisions" [TMD09] - high-level component organisation - protocols for communication, synchronisation and data access - performance - assignment of functionality to design elements - selection amongst design alternatives, ... [GS96] • It is both a process and a product.

Procedural/ Structured Style

Strengths • provides mechanism for modularisation and abstraction: procedures • limited reuse of procedures • procedural decomposition is often quite 'natural' - as in the frequency indexing problem Weakness • Global Data is long considered poor design / programming practice • Change type of data affects all procedures that use it. • No Information Hiding. (Is this worse in dynamically typed language such as Python?) • Unexpected side-effects changes in one location to global data can have unexpected side-effects anywhere in the (large) program.

Implicit Invocation Strengths/ Weaknesses

Strengths: - strong support for reuse - plug in new components by registering it for events - maintenance - add and replace components with minimum affect on other components in the system. Loose coupling. • Weaknesses: - loss of control: » when a component announces an event, it has no idea what components will respond to it » cannot rely on order that these components will be invoked » cannot tell when they are finished • Hence explicit invocation is usually provided as well as implicit invocation. In practice architectural styles are combined.

Object-Oriented Strengths/Weaknesses (Architectural Style)

Strengths: • Data representations can be changed inside individual modules without affecting others. • Reuse better supported because modules make less assumptions about others - looser coupling. Weaknesses: • To interact objects must know each other's identity (in contrast to Pipe and Filter). • Side effect problems: if A uses B and C uses B, then C effects on B can be unexpected to A (and vice-versa). • Complex dynamic interactions - we have seen this in JHotDraw.

KWIC Main Program/Subroutine with Shared Data II

Strengths: • Efficient use of space (and time) - computations share same data • Intuitive appeal - natural solution? • Enhancements based on shared data easily accommodated e.g. removing shifts starting with noise words. Weaknesses: • Change of data representation effects all modules - all modules take advantage of explicit data representation - no information hiding. • Not particularly supportive of reuse - explicit references to data structures and other functions. Tight coupling.

programming style Constraints

Styles defined in terms of constraints - these constraints are similar to the invariants imposed by Architectural Styles. • Constrains the design and implementation of the program. Similar constraints can be imposed at architectural level. • Programming Styles can be mixed - different styles for different parts of program/design. Same for Architectural Styles.

Exercises in programming style

Two lectures based on textbook: Exercises in Programming Style by Cristina Lopes • Book describes design and program for same problem in 33 different styles. • Inspired by a French writer (Quenau) who wrote the same (trivial) story in 99 different literary styles. • Styles range from the common (OO, Functional ...) to relatively rarer Aspects, Restful, ... • We are going to look at 5 styles: Monolithic (traditional), Procedural, ObjectOriented, Pipeline (Pipe and Filter), Holywood (Implicit Invocation) • Video of good talk by Cristina Lopes on Programming Style. Worth watching. https://www.youtube.com/watch?v=Gw4WJJoDl3U

Procedural (cookbook) style in practice

Typically used in smaller programs written in 'procedural languages' such as C (historically Pascal, dialects of Basic). • Still relevant at 'Architectural' Design level where 'global' data is stored in databases and manipulated by well defined functions.

Pipeline (Pipe and filter) III

Unix shell-script programming is closely related - 'pipes' and commands composed together. e.g. ps -ax | grep http • Term frequency can even be solved as a piped sequence of commands at Unix/Linux shell: grep -o "[A-Za-z][A-Za-z][A-Za-z]*" $1 \ # Search for sequences of 2 or more chars | tr '[:upper:]' '[:lower:]' \ # Translate to lowercase | grep -Ev "^($(sed -e 's/,/|/g' ../stop_words.txt))$" \ # Remove stop words | sort | uniq -c | sort -rn | head -25 \ # sort and count top 25 | sed -e 's/^ *\([0-9]*\) *\([a-z]*\)/\2 - \1/' # replaces "2 hi" with "hi - 2" • Pipes are '|' character - connect character stream output to character stream input • Unix / Linux commands are: grep, tr, sed, sort, uniq, head. (Back slash continues line). • Using Unix/Linux, place in a file, make executable, and execute with input file as parameter. No need to understand but feel free to experiment at Unix command line e.g. % echo "I am3 bad" | grep -o "[A-Za-z][A-Za-z][A-Za-z]*" % am % bad

Object Oriented (data abstraction) architectural style

Widely used architectural style • Components: - objects (instances of classes) or abstract data types • Connections: - Method invocation or function/procedure invocations • Invariants: - object preserves integrity of representation - no direct access - representation is hidden from objects • Variations: - objects as concurrent tasks - multiple interfaces for objects (Java) [ Data Abstraction / ADT-based pre-dated Object-Oriented - no inheritance. ]

Architectural Styles or patterns

• Architectural style defines "... a family of software systems in terms of a pattern of software organisation". [GS96] • Architectural style addresses: - Design vocabulary: types of components and connectors - Allowable structural patterns - Underlying computational pattern - Essential invariants of the style - Common examples - Advantages and disadvantages - Common specialisations • Pattern-Oriented Software Architecture [POSA96] by Buschmann et al. describes Architectural Styles as Architectural Patterns (as well as Design and Programming Patterns). Provides thorough guidance on these Styles.

Pipes and Filters (Pipeline) Architectural Style

• Components (Filters) - read streams of data on input producing streams of data on output - local incremental transformation to input stream - output usually begins before input is consumed • Connectors (Pipes) - conduits for streams e.g. first-in-first-out buffer - transmit outputs from one filter to input of other • Invariants - filters must be independent, no shared state - filters don't know upstream or downstream filter identity - correctness of output from network must not depend on order in which individual filters provide their incremental processing • Common specialisations - pipelines: linear sequence of filters - see Programming Style example - bounded and typed pipes ...

Implicit Invocation (Architectural)

• Components: - modules whose interfaces provide a collection of procedures/methods and a set of events that it may announce • Connectors - bindings between event announcements and procedure/method calls - traditional procedure/method calls (to bypass implicit invocation) • Invariants - announcers of events do not know which components will be affected by those events - components cannot make assumptions about ordering of processing, or what processing will occur as a result of their events • Common Examples (Shaw and Garlan [SG96]) - programming environment tool integration e.g. Eclipse IDE (partially) - user interfaces - Model-View-Controller (Observer Design Pattern)

KWIC Main Program/Subroutine with Shared Data

• Four basic functions: Input, Shift, Alphabetise, Output. • Main Program controls these Components and sequences them in turn. • Data is communicated through shared storage: Characters, Index, Alphabetised Index. • Module Input reads the input lines and stores in Characters data structure. • Circular Shift directly accesses Characters to build Index data structure. Each entry in Index identifies the address of circular shift in Characters. • Alphabetiser directly accesses Characters and Index to build Alphabetised Index. This contains an ordered set of indices into Characters. • Output directly accesses Alphabetised Index and Characters to output the ordered, shifted titles.

Heterogeneous Architectures

• In practice the architecture of large systems is a combination of architectural (and programming) styles: - A component in one style may have an internal style developed in a completely different style e.g filter component developed in OO style, implicit invocation module with a layered internal structure. ('Hierarchical heterogeneous') - Overall architecture at same level is a combination of different styles e.g. repository (database) and main-programsubroutine. ('Locational' heterogeneous). Here individual components may connect using a mixture of architectural connectors - message invocation and implicit invocation.

Pipe and Filter: Strengths

• Intuitive - overall behaviour is a simple composition of behaviour of individual filters. • Reuse - any two filters can be connected if they agree on that data format that is transmitted. • Ease of maintenance - filters can be added or replaced. • Prototyping e.g. Unix shell scripts are famously powerful and flexible, using filters such as sed and awk. • Potential for parallelism - filters implemented as separate tasks, consuming and producing data incrementally.

And there's more...

• Lopes identifies 33 different styles and presents Python programs for term frequency in each. • Covered some of the main styles in these lectures. • Some of the remaining styles are rather specialised though there are some key ideas covered e.g. reflection, aspects, plugins, defensive programming, strict typing, persistent / database, concurrent, model-view controller, restful. • We will return to the main styles in the remaining two lectures on Architectural Styles.

Monolithic (Unstructured) Key properties

• One piece of code from beginning to end. • No new abstractions (methods or classes). Little use of libraries. • Lack of subdivision- modularisation. • Little reuse. • One conceptual unit. • Original style from 60s and 70s. • Main weakness: long blocks of program text ('monolithic') with no high level abstractions to aid understanding. • Monolithic lacks constraints on style - no abstractions.

Object Oriented

• Problem decomposed into objects that make sense for problem-domain. • Most widely used style but not necessarily the best for all problems ... hence need to be aware of other styles. • Remember, this is a general description of the key constraints of object-oriented style. • Lopes calls this style 'Things' - she is keen to avoid 'style wars' and actively avoids standard terminology such as OO (watch her video - link on myplace)

Conclusions

• Programming Styles impose constraints on design and implementation (... and 'Architecture'). • Each of the styles have strengths and weaknesses ... some more than others. Benefits (and weaknesses) flow from constraints. • We'll see more of this when we examine Architectural Styles. • Close relationship between Programming Languages and Styles but, as Lopes demonstrates, can often adopt different styles in one programming language. • In practice styles are often mixed e.g. general OO with eventdriven for plug-in flexibility.

And there are more

• Repositories / Data Centred (Database) Central data store component representing systems state and a collection of independent components that operate on the data store. • Interpreter Architecture is based on a virtual machine produced in software. Components are program being executed, its data, the interpretation engine and its state. • Distributed Systems Common organisations for multiprocess systems characterised either by topological organisation e.g. ring or star, and inter-process protocols e.g. clientserver architectures, peer-to-peer. • Process Control Systems Dynamic control of physical processes based on a feedback loop.

Pipeline style (Pipe and Filter) II

• Similar decomposition as cookbook except that all have one input and one output. In general do allow multiple inputs. • No accessing outside state - key constraint of this style. Functions have no side effects. Always produce same output for same input. • Pipeline design often shown as sequence of pipes and filters: • Can also be described using mathematical function composition f o g ("f after g"). Main program at end shows this function composition. • Closely related to more general 'Pipe and Filter' Architectural Style - see next lecture. • Power of this style is that it allows reusable functions to be composed in different ways - highly reusable. (Lopes states: "Haskell leads the way in encouraging pure Pipeline style")

Object Oriented III

• Solution based on decomposition into classes and objects: DataStorageManager, StopWordManager, WordFrequencyManager, WordFrequencyController • Each class has interface through which other classes objects communicate: WordFrequencyManager: __init__, increment_count, sorted • Classes should hide the representation of data structures used - except the data which is passed in and out through interfaces. No "private" instance variables in Python. Convention that name prefixed with an underscore is non-public part of API. Implementation detail, subject to change without notice. • Would be similar in Java ... would static typing help ... ? • Could also have inheritance ...

Layered Systems Strengths/Weaknesses

• Strengths: - increasing levels of abstraction as we move up through layers - partitions complex problems - maintenance - in theory, a layer only interacts with layers above and below. Change has minimum effect. - reuse - different implementations of the same level can be interchanged. - standardisation based on layers e.g. OSI, Java Virtual Machine. • Weaknesses: - not all systems are easily structured in layers - see case studies. - performance - communicating down through layers and back up, hence by-passing may occur for efficiency reasons

Architectural Style: Event Based, Implicit Invocation

• This architectural style (pattern) is characterised by the style of communication between components: Rather than invoking a procedure directly or sending a message a component announces, or broadcasts, one or more events. • Basically, components communicate using a generalised Observer Design Pattern style of communication. AKA Publisher - Subscriber. BUT a different architectural style from OO - communications are broadcast-based and components are not necessarily objects. • Components register interest in an event by associating a procedure with the event. • When the event is announced the system implicitly