Continuous Delivery 3.0
Describe Cloud computing architecture
(API) Software Development Services (API) - - - - - - - - - - - - - - - - - - - - - - - - - - (API) (Appl Serv) (Middelware Serv) (DB Serv) (API) Platform Services (API) - - - - - - - - - - - - - - - - - - - - - - - - - - (API) (Compute) (Storage) (Network) (API) - - - - - - - - - - - - - - - - - - - - - - - - - - (API) Infrastructure Services -Hosting Platform-
Cimprovement - Episodic Change Approach describe Fase Measure downstream
(fases) ... v (mgt) Analysis reported v (key-actv) Study object and generate idea
Cimprovement - Episodic Change Approach describe Fase Mobilize downstream
(fases) ... v (mgt) Goal Defined v (key-actv) Setup teams
Cimprovement - Episodic Change Approach describe Fase Design downstream
(fases) ... v (mgt) Progress and Results communicated v (key-actv) Design object model prototypes
Cimprovement - Episodic Change Approach describe Fase Evaluate downstream
(fases) ... v (mgt) Report delivered v (key-actv) Evaluate archieved result
Cimprovement - Episodic Change Approach describe Fase Implement downstream
(fases) ...t v (mgt) Progress and Results communicated v (key-actv) IImplement Business Model
Steps to move to SaaS based CD
*Enploy PaaS based CD platforms* ++ hide complexity of servers provisioning, configuration, scaling and apps deployment *Employ Saas based CD platforms* ++ usage of SaaS based provider services -- Payment is based on the consumed computing time
CIntel - Name Data Collection Offline Source (offline)
*Human Sources* = { surveys, focus groups, interviews } *Locations* = shopping streets, companies *Physically documented* = { Paper, Magazines, Books}
CI - What do non-continuous integration look likes
*MANUAL* - unit testing - integration - building - deployment *SCM* Classical like ClearCase, Perforce, CVS, Visual Sourcesafe
In Continuous Monitoring what are *Quality* attributes
*NON FUNCTIONALS* - pipeline speed - queue size - delivery time from commit to release - vulnerability footprint - volume usage - server load - CPU usage
CP - Agile Portfolio theory: (Graph_Theory) describes the nodes, links and the attributes
*Nodes* = Backlog items *Links* = Relationships *Attributes* of nodes
In Continuous Monitoring what are *Functional* attributes
*RESULTS* of - unit test - component test *STATUS* of - compoment deployment - micro services - server
CT - Q4, Performance testing
*Test Indicators* - CPU, Memory, I/O, Threads, *Continuous testing* - Setup test based on requirements - Set first baseline - Test un per change, night, sprint *With non cloud solutions: Plan resources* - Test platform - production alike - Test specialist
CT - Q3, tooling
*usability* - eye tracking - heat map (hotjar, mouse movement) *scenario testing - simulators - test data* - selenium *Exploratory testing* - JIRA capture - Q-test Explorer
Containers - what are advantages of containers
++ containers provide the ability to package and run an application in a loosely isolated environment ++ isolation and security allow running many containers simultaneously on a host ++ lightweight without the need of an hypervisor ++ run directory within the host machine's kernels ++ can also be run within virtual hosts
IaC - what are advantages of IaC
++ cost reduction ++ speed (faster) ++ risk (remove errors and security violations)
CP - Agile Portfolio What are the plus and minusses of Hierarchy Planning method?
++ more structured -- implies many dependencies -- less flexible :: Master - slave paradigm :: Machine metaphor :: looks like a piramide
CP - take away
- An Agile portfolio is based on networks and hierachies - Determine dependencies and try to eliminate them - Prioritize based on value, CoD and Risk - Make a list of dimensions and develop Roadmaps and Kanban - Choose between physical and digital boards - Prioritization w AI coming up
IaC - what is the main different between Ansible and Puppet
- Ansible requires no agent on the target nodes - PUSH method - Puppet requires agent on the target nodes - PULL method employed
CI - take away
- CI - definitions - Buildserver - functions and limitations - Version control - type and policies - how to design for CI - build quality in - checks inside the pipelines - pipelines - how to scale - with scaled CI buildserver
What is continuous delivery in the context of CD 3.0
- Continuous *Integration* - Continuous *Testing* - Continuous *Deployment*
CD 3.0 - describes the 3 main Continuous of *Continuous Delivery* area
- Continuous Integration - Continuous Testing - Continuous Deployment
CP - What is WIP Limit (Work In Progress)
- Core properties of a Kanban - limits can be set on person, of team, column - when limits exceeded, team should discuss how to prevent it from happening again
CP - What are the key points of Agile Story Flow way of working
- DONE implies deployed to PRODUCTION - MANAGE your backlog CONTINUOUSLY - BackLogs is a Scrum term - WIP is a Kanban term
CT - Q4, Test goals
- End product and Technology facing - How fast - Performance - How long - stability - How often - reliability - How much - scalability - maintainability - compatibility - instability - testing the Non-Functionals
What is the possible difference between Episodic Change Paradigm en Evolutionary Change paradigm
- Episodic ~ one way ticket to the tropic - Evolutionary - continous improvement loop
What are advantages of infrequent releases versus frequent releases?
- Freq: smaller releases ==> less risk - Infreq: Effort Peaks, High risk
CT - Q4, Performance tooling
- Jmeter - Monitoring tools: perfmon, jconsole, jprofiler - Commercial: HP Loadrunner, Silkperformer - Performance Test Cloud providers
CIntel - List 5 Data Analyzing Methods
- Linking data - Correlation Analysis - Regression Analysis - Monte Carlo - Neural networks
CIntel - what is the google heart framework
- Matrix - Row Happiness, Engagement, Adoption, Retention, Task Success - Column - Goals, Signals, Metrics
CIntel - what is the usage of UTM for marketing campagne
- Monitor various campaigns. - Track audience behaviors. - A/B test your campaigns.
CIntel - what is the context of the correlation and linearity
- Pearson correlation determines the strengh of a lineair correlation. - visual representation is advised to determine OTHER than a lineair representation
CT - Q3, Testing w users/customers
- Sprint review - Potentially shippable products - Build confidence - Short cycled feedback loops - Informal user/ customer demos__ - customer collaboration over contact negotiations - pair exploratory testing - best end use cases
CT - Q4, Security testing, tooling
- Static - Sonarqube -OWASP - open source security project - Commercial tool: Veracode, Fortify
What are the feature of Binary Repository Features?
- Target for in-house deployment - Share modules/components between teams - promote and stage production artifacts - capture and save build history
CIntel - list the typical browser and App Tracking
- URL extensions (Urchin Tracking Module) - Mouse pointer or touch data - History records (Cookies, Flash Cookies) - Finger printing (IP Address, Browser version, hardware info) - Logging
CIntel - List *Linking Data* techniques
- UTM (uchin tracking module) - Nieuwsletter identification - Cookie user tracking - Third party cookie tracking - Finger printing
CIntel - list the Data Collection method in which the user is UNaware of the measurement
- User response = voice, tone, gesture, body language - Attention tracking - task and reaction measurment - user tracking - user record study - diary study
CT - Describe Q2, left above, Test Goals
- business facing tests - provide clarity to customers - story or requirement base - use real business examples - knowing when the customer is happy
CT - Describe Q2, Test Scope
- collaboration w customers - done throughout sprints - no story done, until tested - team responsibility (DBA, DEV) - require specific expertise, not easily done - must be executed by more specialized testers/developers
CD 3.0 - describe the docker stack building
- configuration - - application - - bin/libs - - platform e.g. docker engine - - host operating system - - infrastructure -
CD 3.0 - describe a virtual machine stack building
- configuration - - application - - bin/libs - - guess os- - hypervisor e.g. vmware- - host os - - infrastructure -
CT - Describe Q3, Test Goals
- customer evaluation - test actual user experience - input for backlog management - provide constructive feedback - build relationship with the customer
CP - list the typical properties of an EPIC
- description - performance metrics - estimated value - estimated costs - underlying User Stories
CT - Q3, Usability testing?
- evaluate a product by testing it on users - pick individuals random (Hallway testing) - let these individuals use the products - observe the users - review usage by experts - draw conclusion and take action
CI - list possible Version Control types
- feature branche - team branche - project branche
CIntel - list the Data Collection method in which the user is AWARE of the measurement
- feedback wall - user experience questionaire - co-design activities as prototyping, wire framing and mindmapping - interview
CP - DIFF - What is the key difference between the Hierarchy Planning Method and the network planning method
- hierachy vs. non-hierachy - more structured vs. less structured - more dependency vs. less dependency - less flexible vs. more plexible - master slave paradig vs peer to peer paradig
CI - Software quality - what is an andon cord
- kanban - toyota - by pulling the andon cord you - can raise an alarm, so you can get help - wisdom of the crowd i.s.o. managed inspection
CT - Describe the layer the Ice Cones start with the top?
- manual GUI tests - biggest - automated GUI tests - - automated integration tests - - automated unit tests -
CT - Describe the layer of the Test Piramid start with the top?
- manual gui tests - smallest - automated gui tests- - automated integration tests - - automated unit tests -
CT - Describe Q2, Tooling
- mindmap - white-board - flow diagram - mockups i.e. balsamiq - GUI testtools - Finesse - Selenium - Cucumber - HP QTP, SilkTest
IaC - what is Ansible
- minimal in nature - no agent on nodes - only require ssh and python .- playbooks use easy and descriptive language based on template
How is Continuous Monitoring related to Agility?
- monitoring the operation of CD 3.0 - the operation is monitored by Attributes - Attributes are split into Functionality and Quality categories
CT - Q3, what is scenarios testing?
- process flow - include documentation - use realistic test data * be careful on privacy
CT - Describe Q1, Test Benefits
- quality as a team effort - focuses on verification - increases team velocity - provides an early stage safety net - increase confidence in design - supports refactoring
CT - Q3, what enables A/B Testing in Continuous Intelligence
- real time feedback - enable go/no-go decision - use Configuration Management for toggles.
CI - list possible Version Control policies
- release branching - develop on the mainline - develop on multiple branches - long lived branche - short live branche
CT - What is TDD? Test Driven Development
- relies the repetition of very short Development Life Cycle - originated from Extreme programming - only changes are allowed proven to meet requirements
CT - Q4, Security testing
- review flaws in the security mechanmisms - protects data and functionality - Test integrity, Authorisation and Authentication - Vulnerability Scan - Penetration Test
CI - why is in branching "Simpler is Better"
- simpler is harder to archive - investigate the impact on the system architecture
CT - Q3, what is exploratory testing?
- simultaneous learning, design, test execution - combined test design and test execution - require test and domain specialism - careful observation - critical thinking - diverse ideas, diverse test techniques
CIntel - what are examples of data type using quanTItative data colleaction
- survey results - posts per day, reponses per day - mouse location - finger pressure - time between pages - time between scrolling
CT - Describe Q1, Test Scope Unit tests
- tested in IDE - developer intent - sw program design
CT - Describe Q1, Test Scope Component tests
- tested with stub - architect intent - system design ++ part of CD 3.0 -- continous refactoring +++ decide centralized unit tests of decentralized unit tests
CIntel = CTracking take aways
- there are mamy way to track user data - browsers provide various possibilities - ad brokers and fingerprinting broker further support for user identification - mobile apps provide endless way to collect data - disadvantage of cloud provider is the lack of data ownership - mind privacy rules (GDPR) while tracking users
CIntel - what are examples of data type using quanLItative data collection
- transcription - hand written test - images of typed of printed text - images of models - real world images
CT - Describe Q1, test goals
- unit and component tests - technical tests - code layered in units & components - isolate the test - combined test harder to find root causes
INTRO - what is a piramide of knowlegde
- wish dom - - k no w l e g d e - - i n f o r m a t i o n - - D A T A -
CT -What is Agile Testing?
- working product over documentation - integrated and team approach - intra-sprint agile test activities -- determine test levels and test types -- design test cases -- test and report early in the sprint
CI - what is the disadvantage of long lifed branches
-- Maintenance -- Customization
CP - Agile Portfolio: What are the plusus and minusses of Network Planning method?
-- less structured ++ implies less deplendency ++ flexible :: peer-to-peer paradigm :: organic metaphor :: looks like a real spiderweb
Continuous Improvement area - Value Stream Mapping?
.. is a lean-management method for analyzing the *current state* and designing the *future state* for the series of events that take a product or service from it beginning through to the customer
Continuous Monitoring defined according to wikipedia
.. is the process and technology used to detect *compliance* and *risk* issues associated with an organization's *financial* and *operational* environment.
CD 3.0 - describe the infinity DEV OPS loop
... Plan > Code > Build > Test > Release > Deploy > Operate > Monitor >
What is Canary Testing/Deployment?
... a pattern for rolling out releases to a subset of users or servers. The idea is to first deploy the change to a small subset of servers, test it, and then roll the change out to the rest of the servers. The canary deployment serves as an early warning indicator with less impact on downtime: ...
BTW what is feature toggle? feature flags?
... are basically ways to control the full lifecycle of your features. They allow you to manage components and compartmentalize risk. You can do pretty cool things like roll out features to certain users, exclude groups from seeing a feature, A/B test, and much more.
What is telemetry again in CD context?
... enables fast and continuous feedback from operations to development. A key factor to automating feedback is telemetry. By inserting telemetric data into your production application and environment,
CD 3.0 - What does CD enable
... fast delivery of new software to the market With technology that integrates, tests and distributes software in a fast pace.
Continuous Monitoring definition related to risk managment
... is a *risk management* approach to cibersecurity that maintains an accurate picture of an agencyś security risk posture
What is PaaS?
... is a Cloud computing service that provide users with: - Application Platform (WebLogic) and - Databases as a Service this is equivalent with tradional middleware infrastructure.
What is dark launching (delivery) ?
... is a process where software is gradually or stealthily released to consumers in order to get user feedback and test performance. Code is wrapped in a feature flag or feature toggle which is used to control who gets to see the new feature and when. - Dark launching a business decision
CT - What is BDD?
... is a software development methodology in which an application is specified and designed by describing how its *behavior* should appear to an outside observer
Definition of Continuous Deployment?
... is a software development practice in which environments are setup and target objects are deployed in an automatic way.
DEF again - Continous Deployment defined
... is a software development practice, in which environments are setup and target objects are deployed in a automatic way
CI - Software quality - what is a linter
... is a tool that flags suspicious usage written in a computer language - i.e. can be used to analyses code again style policies
CIntel - what is the usage of canvas fingerprinting
... is a type of "browser fingerprinting" techniques of tracking online users that allow websites to uniquely identify and track visitors using HTML5 canvas element instead of browser cookies or other similar means.
CIntel - what is hotjar heatmap?
... is a visual representation of the clicks, taps and scrolling behavior of your site's visitors / users. There are 3 types of heatmaps: * Click and Tap * Move * Scroll
CIntel - Step 2 - Identify customer needs - the empathy map
... is a way to identify customer needs - it has 6 areas/questions 1 what does she/he see? 2 what does she/he hear? 3 what does she/he say and do? 4 what does she/he think and feel 5 what is the pain? 6 what is the gain?
CP - what is agile portfolio management?
... is dé manier om wendbaar en flexibel te werken met een directie, afdeling of programma. Het stelt je in staat om 1 inzicht te krijgen in wat je samen wil en doet, 2 de verbinding te leggen tussen mensen, ambities en activiteiten, en 3 samen te sturen op wat belangrijk is.
What is IaaS?
... is equivalent to infrastructure and hardware in the traditional (non-cloud) computing method running in the cloud
Continous Improvement defined
... is the *ongoing* effort to *improve* products, services, or processes, delivered by a software driven company
DEF again - Continuous Planning
... is the Automation of Agile planning process to enable backlog item prioritization, refining, allocation and reporting for an agile eco system.
CT - Q3, What is A/B Testing?
... is the act of running a simultaneous experiment between two or more pages to see which performs or converts the best. Despite the name ... , the experiment can be conducted with as many pages as desired. Once you have decided what to test on your landing page - you can create new versions (or variants) of your page to enter into the experiment. Continuous Intelligence = real time feedback - enable go/no-go decision - use Configuration Management for toggles.
CP - Continuous Planning defined?
... is the automation of the Agile planning process, to enable backlog item priotization, refininng, allocation and reporting for an Agile eco-system
CD 3.0 - what is continuous delivery?
... is the automation of the software delivery process, which enables teams to continuously produce software. It aims at building, testing, and releasing software faster and more frequently
CIntel - Continous Intelligence defined
... is the automation of the software user tracking process, the enable SW company in developing software features that add the most value
DEF again - Continuous Intelligence
... is the automation of the software user tracking process, to enable software company in developing software features that add the most value
CD 3.0 - what is continuous delivery 3.0?
... is the automaton of the *full* software engineering cycle, which enables teams to continuously produce software with maximized added value, by use of _tracking_ and _agile planning_. It aims at intelligence, planing, building, testing, and releasing software *faster* and more frequently.
Continous Improvement - what is cycle efficiency
... is the division of working time / total duration
What is Infrastructure as Code?
... is the management of infrastructure in a descriptive model, using the same versioning as DevOps team uses for source code.
DEF again - Continous Testing defined
... is the process of executing automatied tests as part of the software delivery pipeline, to obtain immediate feedback on the business risks associated with a software release candidate.
What is blue/green deployment?
... means to have two identical environments at given time, one of which is active(blue), other as idle(green). The updates are pushed to idle environment and are tested. Then traffic is switched to idle environment(green). Hence idle(green then) becomes active(blue now) now and the previously active ...
What is SaaS
... the provider maintaining software in the cloud and the user running software from their cloud clients over the internet
Define Cloud Computing from an engineering perspective
...is a computing architecture characterized by a large number of interconnected identical computing devices that can scale on demand and that communicates via an IP network
CI - what is the cost of delay?
...is a way of communicating impact of time on the outcome ... has the units of $/time ... combines an understanding of value with how that value leaks away over time
CT - Describe a lightweight testing process?
00 high level scenario definition 01 detailed scenario definition 02 automation test creation 03 automation & manual test excecution 04 final validation 06 goto step 00 and do it all over again
CIntel - list 6 User experience tracking areas
1 *Utility* - function of the system fit the purpose? 2 *Usability* - easy to use 3 *Aesthetics* - visually attractive 4 *Identification* - can I identify myself with the product 5 *Stimulation* - wow experiences 6 *Value* - what is the value of the product for me
CI - Describe a simple CI system?
1 *dev* commit code to SCM 2 *CI* saw trigger and start a build 2 *CI* build = {Compiling, dev test install, deploy, customer facting tests} 4 *CI* give immediately feedback to *dev*
Continuous Improvement - list the 6 steps of Value Stream Mapping?
1 *identify the action* that take place 2 *specify the duration* of these task 3 *specify* which part was actualy work 4 *specify* the amount of time between these steps 5 search for *loopbacks* in the workflow 6 *average* the total working time / total duration
CIntel - Name at least 7 Data Collection Sources (online)
1 App Usage 2 Browser Usage 3 Social Media 4 Email 5 Search Engines 6 Special Interest Groups 7 Press Radio & TV
CD 3.0 describe Continuous Improvement topics
1 CD 3.0 maturity assessment 2 CD 3.0 grow model 3 Value stream mapping 4 monitoring, transparency and feedback 5 engagment factors
CI - Name 3 Continuous Integration sub-thema
1 Checkout and Check in 2 Build 3 Check Code Quality 4 Unit Testing is part of CI but it is a Test Issue 4 Store in Repository
List now the generic steps to setup your CD 3.0 pipeline
1 Configure your IDE 2 Setup CI 3 Setup your CD pipeline 4 Setup your Continous Intelligence services 5 Setup your Continuos Planning services 6 Create Dashboard to monitor your pipeline
CD 3.0 - List the 5 (five) Continuous Delivery 3.0 areas
1 Continuous *Intelligence* 2 Continuous *Planning* 3 Continuous *Integration* 4 Continuous *Testing* 5 Continuous *Deployment*
DEF again - what is Continous Delivery sub area
1 Continuous Integration 2 Continuous Testing 3 Continuous Deployment
CD 3.0 - List 5 Continuous ... thema of Continuous Delivery 3.0
1 Continuous Intelligence 2 Continuous Planning 3 Continuous Integration 4 Continuous Testing 5 Continuous Deployment
Name Continuous Deployment topic
1 Deploy on Acceptance 2 Deploy on Performance Environment 3 Deploy for A/B Testing 4 Deploy to Production
List the SDaaS based CD 3.0 Strategy
1 Determine your CD 3.0 objectives 2 Analyze your current CD Delivery Pipeline 3 Determine the next steps that add value and the related costs 4 Select the CD 3.0 Services that you would like to lease 5 Validate the integration with your Pipeline 6 Include the running value and cost in Continous Monitoring
CIntel - List the 4 linking data techniques
1 Linking data - newsletter identification 2 Linking data - cookie user tracking 3 linking data - Third party Cookie tracking 4 linking data - UTM 5 linking data - finger printing
Name three Binary Repository Types
1 Local - build deployment files 2 Remote - proxy remote services 3 Virtual - aggregate any number of repositories under a single URL
CI - describe Central version Control system
1 Main Server Repository 2 Collaborators update and commit to this server 3 Subversion
CI - describe Distributed version control system
1 Main server repository 2 every collaborator has his own repo 3 collaborator update and commit to his own repo 4 collaborator push and pull to the main server repository 5 git is the most popular distributed repo
Cimprovement - List the Fases of Episodic Change Approach
1 Mobilize 2 Measure 3 Design 4 Implement 5 Evaluate
CIntel - Step 6 - desribes possible technique to test your MVP
1 Poll 2 Smoke test 3 A/B Test 4 Blue/Green Deployment
CIntel - Name 4 Types of Data Collection Sources
1 Qualitative 2 Quantitative 3 Offline 4 Online
CIntel - Name 5 Data Collection techniques
1 Session Reply 2 User Heatmaps 3 Real-time In-App data 4 Crash Recordings 5 Auto-Detect UI Problems
What are advantages of Blue/Green Deployment?
1 Smooth transition from blue to green 2 Enables rapid rollback
CIntel - what are the two main Data Collection methods
1 User is unaware of the measurement 2 User is aware of the measurement
CIntel - How does linking data method - newsletter identification works
1 User registers Full name and Email address 2 Newsletter is sent to users 3 User clicks the email 4 Webserver records the tracking code and IP address
List the steps how to setup Amazon PaaS webservices
1 Using Amazon EC2 services 2 Configure CI server 3 Provisioning Test, Staging, and Production Environments 4 Configure Continuous Delivery Pipe Line 5 Configure Continuous Intelligence Services 6 Configure Continuous Planning Services 7 Create dashboard for the pipeline
List the steps (7) how to install your continuous delivery pipeline in the cloud
1 Using EC2 services 2 Using developer tools, mgt tools 3 Using AWS analytics 4 Install and configure your IDE 5 Configure Continous Intelligence services on EC2 6 Configure Continuous Planning Services on EC2 7 Create Dashboards of the pipeline in Cloudwatch
CD 3.0 which techniques can be employed to drive the improvement
1 Value stream mapping 2 Agile Scan 3 Continuous Monitoring
CIntel - describe Linking data using Third Party Cookie Tracking
1 [browser] request page 2 [webserver] send page with *third party tag* code 3 [browser] execute tag code and collect data 4 [tagcode] send data to [collection server] 5 [data collection server] sends cookie data based on user preference back to [browser]
CI - the delivery team -- members
1 agile team 2 centralized QA team 3 centralized IT Operations team 4 Devivery team
CI - the delivery team - what is responsibilities of the agile team
1 analyze and design 2 development 3 testing and demo
CI - Describe typical CI practices
1 collaboration between teams 2 trunk based development 3 pull integration pain forward 4 automated quality inspection 5 nightly build 6 Automated testing, builds 7 Asynchronous workflow (mass parallel)
CI - how to perform a minor database incremental changes
1 create a new column 2 read on entries in the from the table, the new column is ignore 3 write on new entry a default value is use
CI - how to perform a incremental interface change
1 create a new interface (NUI) 2 client start using the new interface (NUI) 3 make old interface and new interface the same (OUI = NUI) 4 all clients start using the old_ new interface (OUI~NUI) 5 remove the change interface (NUI) this is an incremental process, seems more work than big bang, changes can be perform INDEPENDENTLY
CI - what is a stranger pattern (phase out database)
1 create new software module with a new db 2 create a dispatcher/loadbalancer to dispatch calls either to the old database or to the new db 3 connect the new module to the legacy system 4 connect the new database to the old database
CP - steps to determine visualization goals
1 determine visualization goals 2 choose primary dimensions 3 determine scaling 4 choose secondary dimensions 5 choose hierachical levels 6 plan the boards 7 experiment different setup
CIntel - describe the 6 steps of Lean Product Process
1 determine your target customer 2 identify underserved Customer need 3 Define your value proposition and business model 4 Specify your mininum viable product MVP 5 create your MVP prototype 6 test your MVP with your customer
CI - What is Continous Integration?
1 developer practice 2 keep a working system 3 small changes, slowly growing 4 integrating a least daily on the mainline 5 supported by a CI system 6 Automated testing
CImprovement - list the two change perspectivies
1 evolution change - modelled by Darwin 2 episodic change - happens in short bursts
What is PaaS CD 3.0
1 hardware less platform 2 PaaS based cloud services 3 Hosted by PaaS providers 4 Installation of Open Source products 5 Together a Continuous Delivery 3.0 platform
CD 3.0 - what is maximizing added value means (5)
1 improve customer satisfaction 2 lower risk 3 decrease time to market 4 less required capital 5 increase profit
CI - What are benefits of CI
1 increase transparency 2 increate cooperation 3 increase communication 4 enable people to work on same code
CI - What are benefits of CI?
1 increases transparancy 2 increases cooperation and communication 3 enable people to work on the same code
CI - Agile Architecture characteristics
1 incremental design 2 incorporate change indenpendenly this is how developers work 3 frequent delivery (deliver often)
CT - Selection considerations
1 integration w other tooling & standards 2 compartibility 3 future readyness 3.1 gartner magic quadrant for ref 4 potential test coverage 5 easy of maintenance 6 pricing policy & lock-in
CI - software quality - in software development - when to pull the andon cord
1 linting fails 2 unitest fails 3 integration test fails 4 capacity test fails 5 deploy fails
CT - What are the objectives of Software Testing
1 mimimize the cost of sw as results of bug 2 finding defects of programming the sw 3 gaining confidence and providing information about the sw quality 4 prevent defects other parts of the sw products
What are the objectives of Continuous Deployment?
1 minimize the cost of manual setup and deployment 2 speed up the number of release 3 Improve the reliability of a software release 4 better utilization of the platform and physical infrastructure
CD 3.0 - Ten principes of CD 3.0
1 repeatable, reliable process for releasing sw 2 everything under version control 3 if it hurt, do it more frequently, bring the pain forward 4 automate as much as possible 5 done means released 6 build quality in 7 everybody is responsible for the delivery process 8 use extensive user tracking 9 Plan sw dev based on proven value 10 build a process of continuous improvement
CI - How to handle relationship with compliancy authorities Tracibilty, Payment Card Industry are examples of compliance authorities
1 share your work, own your process 2 share your solution 3 implement their requirements
CI - How do Non-Continuous Integration look like? (8)
1 the same release for a long time 2 long isolated dev. work 3 long live branches 4 integration hell, merging conflict 5 waterfall workflow proces 6 Long acceptance and performance testing 7 Dramatic slowdown of cycle-time when system size increases 8 Manual Unit T, Integration, building deployment 9 Classical version control RCS, ClearCase, SourceSafe
CI - What do non-continuous integration look likes?
1 the same release for a long time 2 long isolated development work 3 long-lived branches 4 Integration hell (Tedious merging conflicts) 5 waterfall oriented working process 6 Long acceptance and performance testing 7 Dramatic slowdown of cycle-time when system size increases
Cintel - describe Linking data - what is Cookie User Tracking
1 tracker javascript code added to the website pages 2 user visit the web page 3 code collect user data 4 unique cookie is generated for an amazon user 5 next visit, amazon recognize user cookie 6 returning visitor and their clicks are recorded 7 information is stored in DB
CIntel - describe linking data - what is Fingerprinting
1 user vists website 2 website request *ad broker* to add an advertisement 3 *ad broker* sent an advertisement and a script to create a *fingerprint* 4 the script reports the fingerprint to the ad-broker 5 the ad-broker reports fingerprint to the *fingerprinting broker* 6 the fingerprinting broker runs the fingerprint against the db and returns the ID to uniquely identify the user 7 the ad-broker uses this info to uniquely track the user on each webpage that show on the advertisement
CI - automate software quality - how to implement
1 using tool 2 scan the code - log warning and errors 3 stop the build when software test fails - andon cord, build breaker plugin 4 inform on deploy status
CT - Describe TDD process?
1 write a test that defines a function 2 run the test and see if the new test failed to validate the test harness 3 write code that cause the test to pass 4 run test to validate code and check that code not delegrate 5 refactor 6 repeat
Starting from the bottom and going clockwise what are the names of the four sides of the Agile testing quadrants?
1. Technology Facing 2. Supporting the Team 3. Business Facing 4. Critique Product
CP - Agile describe the validated learning loop
10 Build 20 deliver Minimal Viable Product experiment 30 Measure w customers and users 40 the results in subjectives and objectives data 50 Learning from the feedback 60 results in Idea 70 goto Build to start it all over again
CP - understand how to calculate Big Bang Cost of Delay
1000 invest of 12 m = $ 12000 interest 5% of 12000 is $600 total 12000 + 600 = $12600 iterative - every month you deliver - so every you make money 1000 x 1.05 for the first month (1000 x 1.05) x 1.05 for the second month etc so you earn 16713 for 12 month the CoD = 16713 - 12600 = $4113
CImpr - What is the Agile Performance Area
3 circles related to each other (Technoglogy & Tooling) - CD 3.0 (People & Culture) (Process & Organisation)
CT - What are Agile Testing quadrants?
4 quadrants (not a gardner quadrants) - support for agile teams and business and technological needs - each quad supports specific test objectives - share responsibilty and collaboration - clarifies required resources in an early stage
CD 3.0 - describe Continuous Deployment topics
> (unitest) > (store) > deploy 2 A> (acceptance test)> deploy > (perf test) > deploy 2 production>
DEF again - Describe the Continous delivery pipeline
> *CI* - Commit > *CI* - Build > *CI* - Check Code Quality > *CT* - Unit Testing > *CI* - Store in repo > *CD* - Deploy on Acceptance > *CT* - Acceptance Testing* > *CD* - Deploy on Performance Environment > *CT* - Performance Testing > *CD* - Deploy on Production
CD 3.0 - describe Continuous Monintoring tooling (example)
> *Track* (hotjar)> *Plan* (JIRA) > *SCM* (bitbucket) > *Build* (maven) > *Integrate* (jenkins)> *quality* (sonar) > *store* (nexus)> *test* (junit|jmeter)> *deploy* (xldeploy)> *operation* (elastic)
List the continuous monitoring tooling again
> *Track* - hotjar > *Plan* - JIRA > *SCM* - Bitbucket > *Build* - maven > *Integrate* - jenkins > *Quality* - sonar > *Store* - nexus > *Test* - junit > *Deploy* - xldeploy > *Monitor* - nagios
CD 3.0 - describe Continuous Testing topics
> *unitest*> (store) > (deploy) > *acceptance test*> (deploy) > *perf test*> (deploy 2 production)>
CD 3.0 - describes now Continuous Testing w the associated environment or tooling
> //Unit testing// *Unit test runner* > Store in repository > Deploy on Acceptance *Acceptance Test Runner* //acceptance testing// > deploy in performance environment *Performance Test runner* //performance testing// deploy to production >
CP - Name 3 Continuous Planning topics
> Agile Epic Planning > Agile Road Mapping > Agile Backlog Planning
CD 3.0 - describe Continuous Planning topics
> Agile portfolio > Planning > Visualization >
CD 3.0 - describe Continuous Montitoring tooling
> Track > Plan > SCM > Build > Integrate > quality> store > test > deploy> operation
CT - Name Continuous Testing topic
> Unit Testing > Acceptance Testing > Performance Testing
CD 3.0 - describes now Continuous Integration w the associated environment
> commit *SCM* > build & document generation *Build & doc generator* > check code quality *code quality checker* > unit testing *unit test runner* > store in repository *component repository*
CD 3.0 - describe Continuous Integration topics
> commit > build > check > (unittest) > store >
CD 3.0 - describe Continuous Intelligence topics
> data collecting > data analysis > result reporting >
CIntel - Name 3 Continuous Intelligence topics
>Data Collecting >Data Analyzing > Result Reporting >
CImpr - What is an Agile Scan
A plot of the 3 Agile Performance Area results in a number.
CP - Continuous Planning Pipeline?
Agile Portfolio > Planning > Visualization
What is the evolutinary source for Agile change
Agile loops drives the 6 continuous areas > CIntel > CPlan > CInteg > CTest > CDepl > COps >
CIntel - what is UTMs (Urchin Tracking Module) works
An ... code placed at the end of a URL. Simply put, UTMs are codes that are placed at the end of a URL. They can track various types of information, including the traffic source, the search terms, and medium (such as email or social media). ... started as a third-party tool, although they're now a part of Google Analytics,
TDD and BDD all rely on what?
Automated testing and effective continuous integration
CP - Agile Portfolio theory - What do you store in Nodes?
Backlog items delivable (large to small) Backlog items Activity (large to small)
What is a Binary Manager?
Binary = artifacts Binary manager manages artifacts Nexus is a Binary Manager
CIntel - what is canvas fingerprinting
Browser usage creates an unique image an unique ID that is semi unique to a user. Tracking code can follow users from website to website. Even when cookies are disabled
CD 3.0 - what are the differences of CD and CD 3.0
CD aims at build, test, release SW CD 3.0 aims at intel, plan, build, test, release SW
CP - Agile Portfolio theory - What do you store in Attributes?
Category (name) required resources (amount) required time (long to short)
Continuous Planning - what is Cost of Delay
CoD is 20000$/minute
CI - What is a scaled CI system
Consist of a network of CI systems LowLevel = *Component Build* CI systems *Feature Build* CI systems in the middle and one big *System Buld* system
Definition of CD by devops.com
Continuous Delivery (CD) is a software strategy that enables organizations to deliver new features to users as fast and efficiently as possible. The core idea of CD is to create a repeatable, reliable and incrementally improving process for taking software from concept to customer.
CT - Continuous Testing Defined
Continuous testing is the process of excecuting automated test as part of the sw delivery pipeline, to obtain immediate feedback of the business risks associated with a sw release candidate
CT - 4Q - Agile testing quadrant - What is critique product?
Critique Product Tests are used for discovering inadequacies in the finished Product - the NON FUNCTIONAL tests
TDD helps to?
Define requirements in test cases Write code to implement them Verify code by running them
CI - how to prioritize EPIC?
Determine Epic BuCa = Estimated Value / Expenses Prioritize Epics based on Buca Slice top Epics in Stories Determine story BuCa Prioritize Stories
Continuos Planning - What is Buca
Epic Business Case = Estimated Value / Expense use for priotization mechanism epic or story
CP - Name Continuous Portfolio Planning hierachy
Epic { Feature { Story { Task }}}}
CIntel - what data can be collected on a mobile App
Everything - Address book, Calendar, Call History, Location, moving censors, .. The app device settings usually allow all by default
What are the elements of Q3?
Exploratory Scenarios Usability Testing User acceptance testing Alpha / Beta Testing Comprise of Manual Testing System or User Acceptance Level, Business Facing and focus on real time scenarios. User Acceptance Tests belong to this quadrant. These tests are manual.
what is the purpose of hotjar Conversion Funnels feature
Find the biggest opportunities for improvement and testing by identifying on which page and at which step most visitors are leaving your site.
What are the "usage" of Finger printing - FirstParty Cookie user tracking and Third party user tracking
Finger printing is used to link user with advertisement First Party Cookie is used to link a frequent visitor to for example amaxon Third Party Cookie is user to link user als first party but the cookie database is not reside on other webserver
What is BDD? Behaviour driven development
Focuses on scenario testing to make sure the system behaves as it should.
What are the elements of Q2?
Functional Testing Examples Story Tests Prototypes Simulations Comprise of Automation and manual testing System level, business facing, and conform product behavior. Functional tests belong to this quadrant. These tests are either manual or automated.
CIntel - What is Hotjar?
Hot jar is an all-in-one analytics & feedback tool that tells you how a website is being used and what users think about it. It visually tracks what visitors do on your site, 1what they are looking for and what they think about it
List type of cloud services
IaaS - Infrastructure as a service PaaS - Platform as a service SaaS - Software as a Service SDaaS - Software Development as a Service
What is the purpose of hotjar recordings feature
Identify usability issues by watching recordings of real visitors on your site as they click, tap, move their cursor and navigate across pages.
what is the purpose of hotjar Form Analysis feature
Improve online form completion rates by discovering when and why your visitors abandon your forms.
CIntel - DIFF - what the different between First Cookie User Tracking AND third party Cookie Tracking
In "third-party cookie", the word "party" refers to the domain as specified in the cookie; the website that is placing the cookie. So, for example, if you visit widgets.com and the domain of the cookie placed on your computer is widgets.com, then this is a first-party cookie. If, however, you visit widgets.com and the cookie placed on your computer says stats-for-free.com, then this is a third-party cookie.
CT - agile testing quadrants, describe the corners
LB - automatic LA - automatic & manual RA - manual RB - tools
IaC - what is Infrastructure as Code?
Managing and Provisioning computer data centers through machine-readable definitions file (= code) rather than physical hardware configuration or interactive configuration tools
What is TDD? Test driven development.
Normally implemented at unit or component test level. Unit tests are repeatable and can be automated.
CP - what are the advantages of online visualisation
Online visualisation less labor intensive + flexible with regards to updates + better for handling distributed team + more secure
What are the elements of Q4?
Performance Testing Load Testing Security testing Scalability Testing Reliability Testing Use of special tools System or Operational Acceptance Level, Technology Facing and Focus on Performance, Load, Stress, Maintainability, Scalability Tests. Special tools can be used for these tests along with automation testing.
CP - Describe the Project Based supply based planning triangle?
Plan driven Given: fix = top = Requirments To be planned: bottom - Resources & Time
CT - Describe the boundaries of these 4 quarter
Q1 - Technology Facing, Supporting the team Q2 - Business Facing, Supporting the team Q3 - Business Facing, Critique product Q4 - Technology Facing, Critique product
CT - Give a summary of Tests being perform in each quarter
Q1 - Unit tests, Component tests Q2 - Functional Tests Q3 - Exploratory Testing, Usability Testing Q4 - Performance and Load Testing, Security Testing
CP - Agile Portfolio theory - What do you store in Links?
Relationship Aggregation (a is part of b) Relationship Dependency (first c, then d)
CT - Describe Q1, Tooling
SCM, IDE, CI, jUnit, xUnit,, CI Tools, mocking tools
What is SDasS
SaaS, Web Based, community shared development tools, eq. SEAT concept, locally installed tools
CP - how does an Agile Road map looks like
Strategic Focus {Goal, Approach, Focus} Time Q1, Q2, Q3, Q4 Contain Milestones Epic User story and value
What is an Andon Cord?
The Cord to stop the production process flow to get help, to troubleshoot the current
The goal of CD by devops.com
The goal of Continuous Delivery is to enable a constant flow of changes into production via an automated software production line. The Continuous Delivery pipeline is what makes it all happen.
What is Episodic Change Paradigm
UNFREZE -> Change -> Refreeze like restructuring like creation of real crisis like perception of a real crisis
CIntel - what is the objective of Continuous Intelligence
Ultimately gaining Wishdom - from Knowlegde from Information - from Data using Machine Learning ie. tensor flow
What are the elements of Q1?
Unit Testing Component Testing Intergration Testing Comprise of Unit and automation testing Unit Level, Technology Facing, and supports the developers. Unit tests belong to this Quadrant. These tests can be Automated tests.
CIntel - what is the usage of cross browser fingerprinting
Until now, however, the tracking has been limited to a single browser. This constraint made it infeasible to tie, say, the fingerprint left behind by a Firefox browser to the fingerprint from a Chrome or Edge installation running on the same machine. The new technique—outlined in a research paper titled (Cross-)Browser Fingerprinting via OS and Hardware Level Features—not only works across multiple browsers, it's also more accurate than previous single-browser fingerprinting.
CIntel - What is cross browser fingerprinting
Use Code to instruct browser to perform a variety of tasks which draw on computer hardware resources, CPU cores etc, install font. This way a browser fingerprint is build with 99% accuracy
CP - when to use feature
Use feature to manage dependencies between teams, matrix agile teams in rows agile teams work on story story which requires teams to realized will be manage by features (column)
CP - Describe the Scrum Based supply based planning triangle?
Value driven Given: fix = top - Resources & Time To be planned: bottom - Features (scope)
CP - what is an agile portfolio?
Wanneer activiteiten en doelstellingen één project overstijgen, bijvoorbeeld omdat je vorm wil geven aan een programma of een organisatieonderdeel, kun je deze onderbrengen in een Agile Portfolio
CT - what is the possible usage of test quadrant
When communicating test ideas, the tester can refer to the matching quadrant so the team will understand the purpose of the test. Answer the question: WHAT-TESTING-WHEN?
CP - how does an kanban board looks like
X-axis Flow of activity Y-axis Activity
CI - what is an integrated development pipeline
[ Commit ] - jira, confluence, bamboo (master, slave) bitbucket, sonar [ Automated Regression test] builder nodes, integration testing, bamboo, artifactory [ Automated Capacitity test] blazemeter, bamboo, artifactory [ Manual test ] computing hosts with manual test cases [ Release ] monitoring with elk, new relic, hosts, configuration mgt with ansible tower enz
CI - list tooling (example) of a development pipeline
[ Commit phase ] ruby on rails, java, npm, junit, ansible, artifactory, jira, confluence, bitbucket, bamboo, artifactory, amazon EC2 instances
Name the elements of a integrated development pipeline?
[ Continuous Integration ] [ Automated Integration Test] [ Automated Performance Test ] [ GUI test ] [ Continuous Deployment ]
CP - describe a typical Story flows
[*Definition of started*] > New > Define [*Definition of Ready*] > Ready > Open > In Progress [*Definition of Done*] > Done
List the main steps of Continuous Delivery
[BUILD] - compile, unit test, version, package [QUALITY] - metrics, documentation [TEST] - acceptance, regression, and performance tests [PROVISION ENV] - deployment to test- and staging environment [PRODUCTION] - green/blue deployment to production
CD 3.0 - describe the continuous event flow, sequence diagram
[Dev ] [SCM] [BUILD] [ACC] [UAT] [REL]
What is the Continuos Delivery Event Flow
a UML sequence diagram describing the Continous [Delivery team] [Version Control] [Build & Unit Test] [Automated Acceptance Test] [User Acceptance Test] [Release]
CI - desribe feature toggles
a way to include of exclude functionalities in your code base using header in C
CT - What is Risk Classifications?
allocate Risk class for each user Story allocate during backlog refinement/poke planning
CT - 4Q - What are business facing tests
are test that answers the questions framed with words from business domain. These are understood by the business experts and would interest them so that behavior of the system can be explained in the real time scenario.
CT - 4Q - what are technology facting tests
are tests that answers the questions framed with words from technology domain. The programmers understand what needs to be implemented based on the clarifications on technology.
CD 3.0 how to delegate improvements
as low as possible in the organization
CP - Span of control of Agile DevOps teams
between checkpoints [*Definition of Ready*] [*Definition of Done*]
CP - Span of control of Agile BizDevOps teams
between checkpoints [*Definition of Started*] [*Definition of Done*]
CT - What is the Boehm's curve?
bugs are more expensive to fix later in the process hence find bugs with the highest risk early in the process priority tests based on priority of feature delivery
CIntel - What are neural network method?
by analyzing mathematical models with a neural network, by recognizing patterns and manipulation data
CI - how to scaled up CI
by implementing a constant flow of new features into production, agile delivery
CI - how to perform incremental database change big time
by splitsing schema to different deployment unit the slide your system vertically into deployment unit so the db changes will not affect your deployment
How to choose between physical and digital boards
choosing between offline and online online is better for distributed team, more secure, good interteam communication, less labor intensive offline is better of handling exceptions, enhancing creatively, group cohesion
CP - what is continous portfolio planning
continuous of sprints defines around a big epics, task are solved in sprints
What is the different between SDaaS on premise and in the cloud
every steps executed on cloud instances
List the main steps of Continuous Delivery in the cloud
exactly the same set, but now everything is executed in the cloud
CI - what is version control flow
gitflow is a fine example of version control flow
CP - what is the power fo graphical visalization on behavior?
group w only mental contraction of elbows increase their elbow strength more than group w doing physical elbow contraction exercises
CL - What are full stack frameworks
i.e. Rails - based on Ruby, is the inspirational grandfather of full stack framework
CI - what does if it hurts, do it more often means
infrequent releases create effort peaks result in high risk result in pain frequent releases will result in less risk, less pain
DEF again - Continuous Integration
is a developer practice keep a working system small changes, slowly growing integrating at least daily to the mainline supported by a CI system
CD 3.0 - what is the purpose of Continuous Improvement
is an area to improve each CD 3.0 area
CT - What are cons of TDD?
management buy in required ... unit test typically written by the developer of the code High nummer of Unit Test may cause false sence of security Badly written unit test very expensive to maintain writing and maintaining excessive number of unit test cost time
CT - What are pros of TDD?
more productive reverting code is more productive that debugging shorter time of implementation ... can lead to more modulized, flexible and extensible code Less code written, easier to automate
CP - what are the advantages of offline visualization
offline vasualization better for group cohesion + better handling of exception + good mental connection w the physical environment of the agile team
CI - the delivery team - what is the responsibility of the centralized QA team
perform the software integration perform acceptance tests perfrom performance tests
CI - the delivery team - what is the responsiblity of the centralized it operation team
perform the software release keep the software up-and-running
CI - the delivery team - what is responsibility of the delivery team
push the changes through the delivery pipeline ( the last mile) to production constantly
CP - study the big bang delivery vs iterative delivery graphs
release at wk 14 vs wk 12 break even at wk 19 vs wk 17
CP - What is a devil triangle?
requirement - Resources - Time triangle are fixed variables no way you can archive the goas -- only 2 can be fixed
CIntel - What is Regression Analysis?
researches the causal relationship between one or more independent variables and a dependent variable
CIntel - What is Correlation Analysis?
researches the relationships between variables
CT - How to classified risk?
risk = probabilty x impact values A, B, C etc select applicable test types for each risk high risks must be covered in the CD 3.0 pipeline
CIntel - What is Monte Carlo method?
simulation method in which a physical process is simulated many times, each time which different starting conditions
IaC - what is imperative programming
telling the machine *HOW* to do something, and as the result what you want to happen will happen.
IaC - What is declarative programming
telling the machine *WHAT* you would like to happen, and let the computer figure out how to do it
Define Cloud Computing from a business Perspective
the cloud is an aggregation of computing services that are scalable and billed on a per usage basis
CP - what is the power of interaction?
the results from sociology shows that the power of social pressure to conform to the group exists
What is Evolutionary Change Paradigm
the validated learning loop again > [BUILD] > *mvp* > [MEASURE] >*subjective & objective data* > [LEARN] > *better ideas* > [BUILD] ...
CI - what is trunk based development
the whole team develops here, not on a shared branche - a CI pattern
Describe the continuous deployment pipeline?
unit testing > store in repository > deploy on acceptance environment > acceptance testing > deploy on performance environments > performance testing > deploy on production
CD 3.0 - what is a value stream map
use for continuous improvement A procedure used in lean manufacturing where the process of disseminating materials and information is streamlined to bring a product or service to the customer by creating the most efficient system possible
Continuous Planning - plot the three lines Cost, Benefit and Profite of Iterative Delivery
x-axis $ y-axis month break even at week 12
Continuous Planning - plot the three lines Cost, Benefit and Profite of Big Bang delivery
x-axis $, y-axis month break even at week 14
CD - layout the elastic - kibana - logstash monitoring framework
{OS. App, Business Logic}.events|logs|metrics are continuously *collected*. An eventrouter handles the data by storing graphing alerting (Elastic+Kibana+logstash) is an possible implementation of the monitoring framework
