COP 4931 Midterm Review

SLO

Service-Level Objective (SLO): SLO is the measurement you set as what is an acceptable level of availability internally. Typically, your SLO is internal and harsher than the SLA — giving you some wiggle room. If your SLO is 99.99 percent availability, your SLA might be 99.9 percent uptime. Set reasonable SLOs. If you aim too high and overstretch your resources, you're setting yourself up for failure because you'll never meet the standard.

Puppet

- Puppet is an open source DevOps systems management tool for centralizing and automating the configuration management process. - It is used to configure, manage, deploy, and orchestrate various applications and services across the whole infrastructure of an organization.

Deleting files in Git

- 'git reset --hard HEAD^' will delete directly from your repo. - 'git reset HEAD <file>' will delete from your index. - 'rm <file>' deletes from your working tree.

"Fast-Forward" Merge

- A fast-forward merge can occur when there is a linear path from the current branch tip to the target branch. - Instead of "actually" merging the branches, all Git has to do to integrate the histories is move (i.e., "fast forward") the current branch tip up to the target branch tip. - This effectively combines the histories, since all of the commits reachable from the target branch are now available through the current one.

Git Repository

- A software repository, or "repo" for short, is a storage location for software packages. - To make the current directory that you are in a repository, type: git init <filename> - Now, check on how things currently are: git status

Apache Ant

- Apache Ant is a software tool for automating software build processes which originated from the Apache Tomcat project in early 2000. - It was a replacement for the Make build tool of Unix, and was created due to a number of problems with Unix's make. - It is similar to Make but is implemented using the Java language, requires the Java platform, and is best suited to building Java projects. - The most immediately noticeable difference between Ant and Make is that Ant uses XML to describe the code build process and its dependencies, whereas Make uses the Makefile format. - By default, the XML file is named build.xml. - Released under an Apache License by the Apache Software Foundation, Ant is an open-source project.

Shrodinger's Canary

- Before modern detection tools came into use, coal miners took canaries into the mines with them. - If poisonous gas began to collect, the canary was the first to die. - Its death warned the humans to the danger and initiated an evacuation of the area. Macabre, to be sure, but effective. - This style of release takes blue-green deploys one step further: It slowly transitions between the two versions rather than cutting over all at one time. - Canary releases ship software changes to select customers as a way of testing functionality and reliability in production while limiting the number of customers potentially impacted.

Blue-Green Deployments

- Blue-green deployments are one of many release styles that seek to reduce service outages resulting from a bad deployment. - In this case, blue and green have no particular meaning. They could just as well have been called pink-red deployments or yellow-purple deployments. This name is simply a way of identifying the two versions of your application running in production. - And that's just what blue-green deployments do — release two versions of your software to the production environment. - You utilize a router to determine which version customers have access to.

Continuous Delivery

- Continuous delivery is a step up from CI in that developers treat every change to the code as deliverable. - However, in contrast to continuous deployment, a release must be triggered by a human, and the change may not be immediately delivered to an end user. - Instead, deployments are automated and developers can merge and deploy their code with a single button. - - By making small, frequently delivered iterations, the team ensures that they can easily troubleshoot changes. - After the code passes the automated tests and is built, the team can deploy the code to whatever environment they specify. - Often, a peer manually reviews code before an engineer merges it into a production release branch.

Continuous Deployment

- Continuous deployment takes continuous delivery even one step further than continuous delivery: every change that passes the entire production release pipeline is deployed. That's right: The code is put directly into production. - Continuous deployment eliminates human intervention from the deployment process and requires a thoroughly automated test suite. - To implement continuous deployment: 1. Maintain a strong testing culture. You should consider testing to be a core part of the development process. 2. Document new features. Automated releases should not outpace API documentation. 3. Coordinate with other departments. Involve departments like marketing and customer success to ensure a smooth rollout process.

Eclipse

- Eclipse is an integrated development environment (IDE) used in computer programming. - It contains a base workspace and an extensible plug-in system for customizing the environment. - Eclipse is written mostly in Java and its primary use is for developing Java applications, but it may also be used to develop applications in other programming languages via plug-ins, including C, C++, C#, JavaScript, and Python.

GIT

- GIT is a version control system which allows you to track changes in your file and, by using it you can easily coordinate the work among your team - Features: DevOps Blog - GIT Logo - Edureka, Free Open Source Tool, Feature Branch Workflow, Allows Distributed Development, Supports Pull Request, Enables Faster Release Cycle

Google's Approach to Service Management

- Google has chosen to run their systems with a different approach: Site Reliability Engineering teams focus on hiring software engineers to their products and to create systems to accomplish the work that would otherwise be performed, often manually, by sysadmins. - Site Reliability Engineering (SRE), as it has come to be defined at Google, is what happens when you ask a software engineer to design an operations team. - A primary building block of Google's approach to service management is the composition of each SRE team.

Beyond the Unit Test

- In unit testing, developers make sure that each component does its job and then continues to do its job after updates and changes. - But what happens when those components get combined? And what happens when they are migrated to the next environment in the pipeline? - Your development life cycle should include time for the following: Developing test cases, Writing automated tests, Running manual tests (if still required), Reflecting on the delivery, and Making adjustments

JIRA

- JIRA is a popular tool for planning developed by Australian Company Atlassian. - It is used for bug tracking, issue tracking, and project management. - The name "JIRA" is actually inherited from the Japanese word "Gojira" which means "Godzilla". - The basic use of this tool is to track issues and bugs related to software and apps. - It is also used for project management. - The JIRA dashboard consists of many useful functions and features which make handling of issues easy.

JUnit

- JUnit is a unit testing framework for the Java programming language. - JUnit has been important in the development of test-driven development, and is one of a family of unit testing frameworks which is collectively known as xUnit that originated with SUnit - JUnit is linked as a JAR at compile-time. - A research survey performed in 2013 across 10,000 Java projects hosted on GitHub found that JUnit, was the most commonly included external library.

Automating Testing

- Manual testing is becoming obsolete. - Our systems and codebases are simply too complex and run in too many different types of environments for a human to confirm that everything works as expected. - Continuous integration requires an automated test suite that runs tests every time code is committed to a code tracking system. - This approach requires not only that your team writes tests but also that you treat your test code as code. - Automation is key to enabling a "shift-left" mentality. Done well, testing allows you to fail early and often.

Practices

- Put simply, SREs run services—a set of related systems, operated for users—and are ultimately responsible for the health of these services. - Successfully operating a service entails a wide range of activities: developing monitoring systems, planning capacity, responding to incidents, ensuring the root causes of outages are addressed, and so on. - We now talk about the theory and practice of an SRE's day-to-day activity: building and operating large distributed computing systems. - We can characterize the health of a service from the most basic requirements needed for a system to function as a service at all to the higher levels of function taking active control of the direction of the service rather than reactively fighting fires. - This understanding is so fundamental to how Google evaluates services that it wasn't explicitly developed until a number of Google SREs needed a way to explain how to increase systems' reliability.

PyCharm

- PyCharm is an integrated development environment (IDE) used in computer programming, specifically for the Python language. - It is developed by the Czech company JetBrains. - It provides code analysis, a graphical debugger, an integrated unit tester, integration with version control systems (VCSes) - PyCharm is cross-platform, with Windows, macOS, and Linux versions.

Regression Tests

- Regression testing verifies that after you make changes to the code, key metrics for how your application works and runs haven't changed as well. - This verification includes previous functionality. Have old bugs resurfaced? Did a new change impact a previous version of an API? - This testing might check that the accuracy or precision hasn't degraded. - Sometimes regression tests are as simple as ensuring that a simple CSS color change didn't make the site a different color or cause a link to break. - Regression testing can be expensive to do: Regression testing often requires that a company already has automated tests in place - they just run their old tests to make sure that nothing has broken.

Selenium

- Selenium is an open source tool which is used for automating the tests carried out on web. - Only testing of web applications is possible with Selenium. You can neither test any desktop (software) application nor test any mobile application using Selenium. - Since Selenium is open-source, there is no licensing cost involved, which is a major advantage over other testing tools. - Other reasons behind Selenium's ever growing popularity are: 1. Test scripts can be written in : Java, Python, C#, PHP, Ruby, Perl, & .Net. 2. Tests can be carried out in any of these OS: Windows, Mac, or Linux. 3. Tests can be carried out using any browser 4. It can be integrated with tools such as TestNG& JUnit for managing test cases. 5. It can be integrated with Maven, Jenkins & Docker to achieve Continuous Testing

Continuous Integration

- Teams that practice continuous integration (CI) merge code changes back into the master or development branch as often as possible. - CI typically utilizes an integration tool to validate the build and run automated tests against the new code. - The process of CI allows developers on a team to work on the same area of the codebase while keeping changes minimal and avoiding massive merge conflicts.

Architecting Code for the Six Capabilities of DevOps

- The architecture set at the beginning of a project ripples out into the decisions made down the road. - The framework determines to what degree the system is flexible for changes and how limited it will be to the addition of certain elements. - Generally, the architects will consider and weigh the six key categories of DevOps capabilities: Maintainability, Scalability, Security, Usability, Reliability, and Flexibility

A Release

- The deployment process refers to releasing that code to customers. - That process can be as simple as clicking a button or as complex as a series of pipelines through which the code must pass to reach customers. - Sometimes you hear the term release used interchangeably with deployment. - A deployment is the movement of code from one environment to another. A developer can deploy their code from their local machine to the development (DEV) environment. - At that point, the code may pass through several more environments, like user acceptance testing (UAT) or quality assurance (QA), staging, and production (PROD). - The deployment to production — specifically, a deployment to customers — is the purest form of the word release.

Rolling the Dice

- The final type of release we will highlight is called a rolling deployment. - Instead of releasing a new version to select customers in small incremental chunks, rolling deployments replace the version of an application running on a specific instance. - The new version is deployed to each instance one at a time (or in clusters) until all instances or machines are running the latest version. - Some companies choose to implement rolling deployments by cutting over multiple machines at the same time. - The size of your grouping is referred to as the window size. - A window size of one will proceed one machine at a time whereas a window size of four will deploy the new version to four servers at the same time.

Four Golden Signals

- The four golden signals of monitoring are latency, traffic, errors, and saturation. - If you can only measure four metrics of your user-facing system, focus on these four: - Latency is the time it takes to service a request. It's important to distinguish between the latency of successful requests and the latency of failed requests. It's important to track error latency, as opposed to just filtering out errors. - Traffic is a measure of how much demand is being placed on your system, measured in a high-level system-specific metric. - Errors are the rates of requests that fail, either explicitly (e.g., HTTP 500s), implicitly, or by policy (for example, "If you committed to one-second response times, any request over one second is an error"). - Saturation is how "full" your service is. A measure of your system fraction, emphasizing the resources that are most constrained (e.g., in a memory-constrained system, show memory; in an I/O-constrained system, show I/O). - If you measure all four golden signals and page a human when one signal is problematic (or, in the case of saturation, nearly problematic), your service will be at least decently covered by monitoring.

Git 'diff' command

- The git diff command is used to compare the files that are in your workspace to files that are either in the index or in the HEAD. The changes between the copies of each file are then printed on the screen. - 'git diff' will compare your working tree and your index. - 'git diff HEAD' will compare your working tree and your repo. - 'git diff --staged' will compare your index and your repo.

Integrating and Delivering Continuously

- The growth of DevOps culture has changed the way developers build and ship software. - Before the Agile mindset emerged, development teams were assigned a feature, built it, and then forgot about it. - Operations was responsible for deploying and maintaining the code in production. - This process was clumsy, to say the least, and it caused quite a bit of conflict. - Because teams existed in silos, they had little to no insight into how other teams operated, including their processes and motivations. - CI/CD, which stands for continuous integration and continuous delivery (or deployment), aims to break down the walls that have historically existed between teams and instead institute a smoother development process.

The New Development Life Cycle

- There are 8 stages to a complete DevOps solution: Plan, Code, Build, Test, Release, Deploy, Operate, and Monitor. - The most important benefit of the development pipeline is that it invites everyone to the table. - It gives everyone the opportunity to get involved as they see fit and to learn new skills if they're interested.

Managing Risk

- To make this problem tractable and consistent across many types of systems Google runs, they focus on unplanned downtime. - For most services, the most straightforward way of representing risk tolerance is in terms of the acceptable level of unplanned downtime. - Unplanned downtime is captured by the desired level of service availability, usually expressed in terms of the number of "nines" they would like to provide: 99.9%, 99.99%, or 99.999% availability. - Each additional nine corresponds to an order of magnitude improvement toward 100% availability. - For serving systems, this metric is traditionally calculated based on the proportion of system uptime: availability = uptime / (uptime + downtime)

Git Commit

- Type the command: git commit -m "Commit Name" - To check if the commit was successful: git log - When we commit a file, we create a copy of it and we place that copy into git's local repository (HEAD). 'commit -a' command will automatically commit files if they are already known to the local repo (does not work for new files).

Git Version Control

- Version control is a system that records changes to a file or set of files over time so that you can recall specific versions later. - It allows you to revert selected files back to a previous state, revert the entire project back to a previous state, compare changes over time, see who last modified something that might be causing a problem, who introduced an issue and when, and more. - Using a VCS also generally means that if you screw things up or lose files, you can easily recover. In addition, you get all this for very little overhead.

Checking Out Branches

- We check out SDN using the command: 'git checkout SDN' - Our labeled commit graph shows that HEAD is now pointing to SDN. - The command 'git status' shows the same:

Creating Branches in Git

- We use the 'git branch' command to create two new branches. - The 'git branch' command by itself will show us all of our branches. - '*' means that the master branch is checked out and head is pointing to the master branch.

Rebase

- We want to merge branch on top of master, but we don't want any merge commits and we want a nice, simple linear master. - The first thing that rebase will do is take the branch pointer and move it to master. The branch commits are still held on to.

Infrastructure as Code (IaC)

- When developers begin to think of infrastructure as part of their application i.e as infrastructure as code (IaC), stability and performance become normative. - Infrastructure as Code is the process of managing and provisioning computing infrastructure (processes, bare-metal servers, virtual servers, etc.) and their configuration through machine-processable definition files, rather than physical hardware configuration or the use of interactive configuration tools. - This is where automation tools play a major role.

DevOps can:

1. Reduce interpersonal friction 2. Eliminate bottlenecks 3. Improve collaboration 4. Increase job satisfaction through engineer empowerment 5. Accelerate team productivity

Three Stages of Git

1. Working Tree 2. Staging Area (Index) 3. Repo

CALMS stands for:

Culture: The best DevOps cultures are extremely collaborative and cross-functional, with people from different teams working together. Automation: Rote tasks are an engineer's worst nightmare, not only because they're boring but also because they're inefficient. Lean: DevOps teams are agile and scrappy. Lean teams avoid low-impact activity because it doesn't provide value to the customer. Measurement: Data is critical to DevOps. Measuring progress through data will inform nearly every aspect of your organization's transformation. Sharing: DevOps creates an environment in which both teams teach each other and feel empowered — thus building a single team.

What is the difference between DevOps and Agile?

DevOps is a culture that allows the development and the operations teams to work together. This results in continuous development, testing, integration, deployment, and monitoring of the software throughout the lifecycle. Agile is a software development methodology that focuses on iterative, incremental, small, and rapid releases of software, along with customer feedback. It addresses gaps and conflicts between the customer and developers.

What is DevOps?

DevOps is an engineering culture of collaboration, ownership, and learning with the purpose of accelerating the software development lifecycle from idea to development. DevOps is no silver bullet, but it can have massive impact on an organization and on products.

JENKINS

Jenkins is a continuous integration server written in Java. You can use it for testing and reporting changes in near real-time. Being a developer, it will help you to find and solve bugs in your code rapidly and automate the testing of their build. Features: - Free Open-Source Tool - Integrate all your DevOps stages with the help of around 1000 plugins - Script your pipeline having one or more build jobs into a single workflow - Easily start your Jenkins with its WAR file - Provides multiple ways of communication: web-based GUI, CLI and REST API

SLA

Service-Level Agreement (SLA): Availability — your service being up and running — is key to success in operations. Customers have to be able to access your applications. In other words, is the application functioning as expected? A service-level agreement is the availability you agree to maintain with your customers over a set period of time. It is typically part of your service contracts and is legally enforceable.

SLI

Service-Level Indicator (SLI): This is the medium through which you measure your success at meeting your SLOs. The indicators are feedback from your systems that give you reasonable insight into your actual availability percentage in each service you measure. If SLIs dip below expected thresholds (SLOs), you need to dig in further and determine whether changes to the system are necessary.