Software Engineering
What are Graph-oriented Databases? What are their benefits and drawbacks? When are they used?
A graph type database stores entities as well the relations amongst those entities. The entity is stored as a node with the relationship as edges. An edge gives a relationship between nodes. Every node and edge has a unique identifier. Each node represents an entity (a person, place, thing, category or other piece of data), and each relationship represents how two nodes are associated. This general-purpose structure allows you to model all kinds of scenarios - from a system of roads, to a network of devices, to a population's medical history or anything else defined by relationships. When are they used? Graph databases are built for use with transactional (OLTP- Online Transaction Processing) systems and are engineered with transactional integrity and operational availability in mind. Use-Cases: Fraud detection, Real time recommendation engines, Master Data Management, Network and IT Operations, Identity and access management Benefits: - Performance - Flexibility - Agility Drawbacks: - Graph databases are not optimized for large-volume analytics queries - Graph databases do not create better relationships - Graph databases are not as useful for operational use cases because they are not efficient at processing high volumes of transactions
Explain the term Enterprise Service Bus.
An Enterprise Service Bus is used to "provide services via a data bus in a corporate network". Properties of ESB -MW manage interactions between services and users -MW hat auch APIs implementiert Characteristics - Optimizes development - Performs data transformation - Provide intelligent routing - Provides real time monitoring - Ensure service security - Mediation/Vermittlung between service consumer(requester) and service provider The complete exchange of messages findet thus takes place via one or more ESBs. It is programming language and platform independent, so that e.g. Java and .NET applications can communicate with each other. Tasks: - Transformation of different architectures - Routing and converting of messages - Coordination of Business processes - Queueing and buffering Advantages - scalability Drawbacks - Additional Middleware
What are the benefits and downsides of a Microservices Architecture?
Benefits: Scaling Changes Fault Isolation No vendor or technologylock Ease of Understanding Drawbacks: Remote Calls Decentralized data Can be complex Testing an application can be difficult Deploying micro-services can be complex Monitoring Benfits Detail: Scaling: Scaling requires scaling of the entire application rather than parts of it that require greater resource. Microservices make horizontal scaling much easier. Changes in a monolith require the entire monolith to be rebuilt and deployed. Microservices can be deployed separately Improve fault isolation: Larger applications can remain mostly unaffected by the failure of a single module. (A consequence of using services as components, is that applications need to be designed so that they can tolerate the failure of services) Eliminate vendor or technology lock-in: Microservices provide the flexibility to try out a new technology stack on an individual service as needed. There won't be as many dependency concerns and rolling back changes becomes much easier. With less code in play, there is more flexibility. Ease of Understanding: With added simplicity, developers can better understand the functionality of a service. Drawbacks Detail: Remote Calls: Remote calls are more expensive than in-process calls. Additional traffic is necessary and the response can take longer. High performance demands can make a strict microservice architecture infeasable. Decentralized data: decentralicing responsibility across microservices has implications for managing updates. Using transactions helps with consistency, but imposes significant temporal coupling, which is problematic across multiple services. Distributed transactions are difficult to implement. Can be complex: Since everything is now an independent service, you have to carefully handle requests traveling between your modules. In one such scenario, developers may be forced to write extra code to avoid disruption. Over time, complications will arise when remote calls experience latency. Testing an application can be difficult: In a monolithic approach, we would just need to launch our WAR on an application server and ensure its connectivity with the underlying database. With micro-services, each dependent service needs to be confirmed before testing can occur. Deploying micro-services can be complex: They may need coordination among multiple services, which may not be as straightforward as deploying a WAR in a container. Monitoring: Since services can fail at any time, it is important to detect failures as fast as possible and automatically restore the service. In order to do this there is a lot of emphasis on real-time monitoring of the Microservices. (business relevant metrics, dashboards etc.)
What are Column-oriented Databases? What are their benefits and drawbacks? When are they used?
Column-oriented databases work on columns and are based on BigTable paper by Google. Every column is treated separately. Values of single column databases are stored contiguously. When are they used? The real value in using column-oriented database technology comes from high performance, scalable storage and retrieval of large to massive datasets (Big Data) focused on aggregation queries. They deliver high performance on aggregation queries like SUM, COUNT, AVG, MIN etc. as the data is readily available in a column. Benefits: - Efficient storage and data compression - Fast data loads - Fast aggregation queries - Simplified administration & configuration Drawbacks: - Transactions are to be avoided or just not supported - Queries with table joins can reduce high performance - Record updates and deletes reduce storage efficiency - Effective partitioning/indexing schemes can be difficult to design
What is Containerization? How does it help to build a Microservices Architecture?
Containerization allows you to run applications in a virtual environment by storing all system components (operating system, application server database systems, and libraries) in a single container. Everything necessary for the execution of the application is located in the container and thus different services can effectively share a kernel of the operating system. Docker Engine bundles an application that can be easily transported to other machines or operating systems. How does it help to build a Microservices Architecture? - Can be deployed quickly (within milliseconds) - They can be scaled up and down easily - They can be duplicated quickly. - Containers with problems can be insulated easily and quickly. - Containers need less CPU and memory than VMs. - Better deployment process - Better automation / Automation - Better agile work - Better and faster release cycles
What are Key-Value Stores? What are their benefits and drawbacks? When are they used?
Data is stored in key/value pairs. It is designed in such a way to handle lots of data and heavy load. Key-value pair storage databases store data as a hash table where each key is unique, and the value can be a JSON, BLOB(Binary Large Objects), string, etc. Benefits: - almost no need to maintain indexes - horizontally scalable - extremly fast - suited for problems where data are not higly related Drawbacks: - lacking indexes - scanning capabilitties
Explain the term DevOps. How does it relate to Microservices?
DevOps is a combination of mindsets, practices and tools that enable organizations to deliver applications and services faster and easier. Development and operations teams no longer work in isolation from each other. They work closely together through the various phases: - Development - Test - Operation DevOps Methods - Continuous integration - microservices - Continuous delivery - Communication between the teams How does it relate to Microservices? Microservices enable DevOps teams to develop independent functions in parallel. Rather than moving code from one specialist to another (eg, development, testing, production), cross-functional teams work together to create, build, test, publish, monitor, and maintain the applications. The modularity of microservices enables incremental releases which is a goal of DevOps. Benefits of Combining DevOps and Microservices: - More frequent software releases - Better quality code - Software reusability - Scalability - Resilience - Higher levels of automation - Flexibility (the ability to consume new data - sources, frameworks, libraries and other resources) - Freedom (different teams can use their favorite - tools and languages) - Cloud-based architecture - Higher levels of DevOps productivity
What are Document Stores? What are their benefits and drawbacks? When are they used?
Document-Oriented NoSQL DB stores and retrieves data as a key value pair but the value part is stored as a document. The document is stored in JSON or XML formats. The value is understood by the DB and can be queried. Benefits: - Easy to getting started - Free schema (can change the schema easily) - Add nodes on the fly with advantage of scalability (mongo detect them as you add) - Rich set of client libraries - Uses BSON (superset of JSON which is easy to deal with) - Great speed if your inserts are not failsafe (which is on by default - use case logging) - Indexing fields is easy (if you need speed at some field just index it, mongodb allows you do that easily) - Geospatial indexing and querying - Evolving very fast Drawbacks: - No joins - Less flexible queries - For complex jobs you need Map-Reduce - May face with unexpected failures (generally not mongo's fault - wrong setup,config etc.) - You are good as long as your index fits into memory (memory mapped files) - Using single node is dangereous (you may lost your data) - Evolving very fast - No builtin fulltext search (comes with latest version but not yet battle tested)
How can Microservices interact with each other? What are the benefits of using Message Queues in a Microservices Architecture?
HTTP, REST or APIs are a popular way of performing RPC between services. Its main benefits are simplified setup in the beginning and relative efficiency in sending messages. Another way of building microservices communications revolves around the use of a message bus or message queuing systems. Old-fashioned Service-Oriented Architecture called these enterprise service buses (ESBs). Benefits: - Asynchronous through Messaging - Security matters are slightly simplified - Centralization increased security - Can use Publish & Subscribe mechanism (ability to totally decouple senders from receivers) Drawbacks: -Many message brokers bad scalability limitations - How message queues behave in error cases - Network latency increases
Define the term cloud computing and explain the following terms: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), Public Cloud, Private Cloud, Community Cloud, Hybrid Cloud.
IaaS Infrastructure as a Service (IaaS). The capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, and deployed applications; and possibly limited control of select networking components (e.g., host firewalls). Platform as a Service (PaaS) The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment. Software as a Service (SaaS) The capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through either a thin client interface, such as a web browser (e.g., web-based email), or a program interface. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings. Private cloud. The cloud infrastructure is provisioned for exclusive use by a single organization comprising multiple consumers (e.g., business units). It may be owned, managed, and operated by the organization, a third party, or some combination of them, and it may exist on or off premises. Community cloud. The cloud infrastructure is provisioned for exclusive use by a specific community of consumers from organizations that have shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be owned, managed, and operated by one or more of the organizations in the community, a third party, or some combination of them, and it may exist on or off premises. Public cloud. The cloud infrastructure is provisioned for open use by the general public. It may be owned, managed, and operated by a business, academic, or government organization, or some combination of them. It exists on the premises of the cloud provider. Hybrid cloud. The cloud infrastructure is a composition of two or more distinct cloud infrastructures (private, community, or public) that remain unique entities, but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds)
Explain WSDL. How is a WSDL 1.1 interface structured? What is defined in the "abstract part" and what in the "concrete part"? Also provide an example of a WSDL 1.1 interface.
In WSDL, specifications are XML documents that describe Web services and in particular describe service interfaces and the access mechanism. WSDL Interface consists of an abstract part (conceptually analogous to conventional IDL) and a concrete part (defines protocol binding and other information). The Abstract part consists of: - Types - Data Types (Strings, Integers, ...) and structures - Messages - that build on the above data structures. - Operations - There are 4 basic operations -One-way -Notification -Request-response -Solicit-response - Port types - Grouping operations into port types The concrete part consist of: - Interface Bindings - Specifies the message encoding and protocol bindings for all operations and messages defined in a given port type. - Ports (EndPoints) - Services - are logical grouping of ports
What are Microservices? What are characteristics of a Microservices Architecture?
In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. So micro-services - also known as the microservice architecture - is an architectural style that structures an application as a collection of services that are: - Highly maintainable and testable - Loosely coupled - Independently deployable - Organized around business capabilities. The micro-service architecture enables the continuous delivery/deployment of large, complex applications. It also enables an organization to evolve its technology stack. The characteristics of Microservices: - Componentization via Services - Organized around Business Capabilities - Products not Projects - Infrastructure Automation
Explain JSON. What are the advantages and disadvantages of JSON compared to XML?
JSON (JavaScript Object Notation) is a compact data format and simple to read and write. The purpose of JSON is to exchange Data between applications Vorteile JSON JSON supports Natvie objects JSON is data oriented JSON provides scalar data types JSON has the ability to present structured data trough Arrays and objects Less words JSON uses name and value pairs with { } [ ] for arrays And : to describe values Less Size With the same content JSON is usually smaller compared to XML JSON is serialized and deserialized faster than XML JSON is similar to JAVA Script that's why the coding and parsing is very similar. Vorteile XML XML is document oriented XML requires tags to open and close XML has a big advantage, it can include metadata in the tags < > of attributes The tree of XML structure allows the Users to simply fold in and fold out. Mixed content support Xml can communicate mixed content via the same data payload The content is clearly separated with markup tags
Explain the concept of continuous integration/continuous delivery (CI/CD). Why is CI/CD important for Microservices?
Mithilfe der Continuous Integration (CI) können Entwickler ihre Codeänderungen in einem gemeinsamen „Branch" oder „Trunk" der Anwendung viel häufiger zusammenführen. Sobald die Änderungen eines Entwicklers zusammengeführt werden, werden sie in automatischen App-Builds und unterschiedlichen Stufen von Automatisierungs Prüfungen validiert. Wenn die automatische Prüfung Konflikte zwischen aktuellem und neuem Code erkennt, lassen sich diese mithilfe von CI schneller und häufiger beheben. Continuous delivery The goal is that an application is always deliverable /production-ready after passing the automated tests. Application is always deployable - Methods such as - Acceptance test Manual deployment to deliver software in a production-like environment Why is CI/CD important for Microservices? Faster release cycles are one of the major advantages of microservices. But without a good CI/CD process, you won't achieve the agility that microservices promise.
What are NewSQL databases?
NewSQL is a class of relational database management systems that seek to provide the scalability of NoSQL systems for online transaction processing (OLTP) workloads while maintaining the ACID guarantees of a traditional database system. The ultimate goal of NewSQL is to deliver a high performance, highly available solution to handle modern data, without compromising on data consistency and high-speed transaction capabilities.
What are RESTful Web Services? How are these different from SOAP based web services?
REST is not a product or standard. REST describes how Web standards can be used in a Web-compliant manner. Basically, everything in a REST architecture revolves around resources. RESTful Requirements: -Addressability Each resource must be identified by a unique resource identifier (URI). -Stateless Statelessness: The communication between the participants is stateless. This means that there are no user sessions (e.g. in the form of sessions and cookies), but that all necessary information must be re-sent with each request. -Uniform Interface Each resource must be accessible via a uniform set of standard methods. Examples of such methods are the standard HTTP methods such as GET, POST, PUT, and more. -Decoupling of resources and representation This means that different representations of a resource can exist. For example, a client can explicitly request a resource in XML or JSON format.
What are NoSQL databases? How do they compare to relational databases?
NoSQL is a non-relational DMS, that does not require a fixed schema, avoids joins, and is easy to scale. NoSQL database is used for distributed data stores with humongous data storage needs. NoSQL is used for Big data and real-time web apps. Non-relational - NoSQL databases never follow the relational model - Never provide tables with flat fixed-column records - Work with self-contained aggregates or BLOBs - Doesn't require object-relational mapping and data normalization - No complex features like query languages, query planners Referential integrity joins, ACID Schema-free - NoSQL databases are either schema-free or have relaxed schemas - Do not require any sort of definition of the schema of the data - Offers heterogeneous structures of data in the same domain Simple API - Offers easy to use interfaces for storage and querying data provided - APIs allow low-level data manipulation & selection methods - Text-based protocols mostly used with HTTP REST with JSON - Mostly used no standard based query language - Web-enabled databases running as internet-f facing services Distributed - Multiple NoSQL databases can be executed in a distributed fashion - Offers auto-scaling and fail-over capabilities - Often ACID concept can be sacrificed for scalability and throughput - Mostly no synchronous replication between distributed nodes Asynchronous Multi-Master Replication, peer-to-peer, HDFS Replication - Only providing eventual consistency - Shared Nothing Architecture. This enables less coordination and higher distribution.
How do you secure the access to RESTful Web Services?
RESTful Web-Services is just a concept! By means of all these variants (protocols) which are already known from web pages; Secure Methods: TLS These protocols provide a secure transmission tunnel between the http client and the http server so that the message cannot be read or modified during transmission. AUTH The client must authenticate itself once to an authentication server because of the statless definition there is no session(cant be used for Auth) JWT (Json Web Token) This token is then sent within the authentication header to the RESTful service, where it can be verified for validity. WAF A web application firewall is a procedure that protects web applications from attacks using the hypertext transfer protocol http. In contrast to classic firewalls, a WAF examines the communication on the application layer. HTTPS API Keys An API key is a code passed by applications that call an API to identify the calling application for the website. Often ( unique identifier and secret token) Cookies Text information that the visited web page or server places on the computer of the viewer (client) via the web browser. The task of a cookie is to identify the surfer, e.g. store the login or an online shopping cart or for web tracking.
How do Microservices compare with SOA?
SOA - SOA-Modell has more ESBs Enterprise Service Bus dependencies/Abhängigkeiten - Use imperative programming - Relational database/relationale Datenbank Microservices - Use faster messaging mechanism - Responsive programming - NoSQL or Micro SQL Databases
Explain the term Service Oriented Architecture (SOA). Differentiate between SOA and Web Services. What does the basic web service infrastructure look like?
SOA(Service Oriented Architecture) : components exchange information through web services. Web services are either designed using RESTful style or SOAP style. Each Interaction is self-contained and loosely coupled. They are independent of other interactions. Web services based on the Simple Object Access Protocol (SOAP) are now the most common Form of SOA.
Explain SOAP. What is the structure and content of a SOAP 1.2 message? What is the difference between document-style interaction and RPC-style interaction? How are SOAP 1.2 messages processed?
SOAP: Simple Object access protocol SOAP defines how to organize information using XML, that it can be exchanged between peers. SOAP is stateless and one-way; ignores semantics of message being exchanged. SOAP 1.2 Message Soap exchanges information using messages. Each message contains 2 parts, a header and a body. Header is optional, body is mandatory. Header and body can have multiple sub-parts in form of header blocks or body blocks. SOAP assumes that every message has a sender, an ultimate receiver and an arbitrary number of nodes that process the message and route it to the receiver. Document style: the 2 interacting applications agree on the structure of exchanged documents e.g. client orders goods from supplier and creates a PurchaseOrder document as a soap message. body of the message has: items + quantities. Header has contextual information. Supplier replies with ACK document that contains the orderID. Usually sent async. RPC-style: One soap message encapsulates the request while another message encapsulates the response, just as in doc-style interaction. Difference is how the messages are constructed. The body of the request message contains the actual call. (includes: name of the invoked procedure and input params). Body of the response has the result + output parameters. Processing of SOAP Message In the SOAP model, nodes processing a message can play 1 or more roles. Each block in a SOAP header may include its intended role. None: such block should not be processed by any node receiving this message Next: every node receiving the message can process that block. (ultimate Receiver is also included in next) ultimate Receiver: block is solely intended for the recipient of the message.
How can Web services be versioned?
Sector-driven versioning Specialist driven versioning leads to the possibility of multiple versions of a service at the same time in the same runtime environment. Trivial business versioning Any change to an existing service is (technically) considered a new service. Of course, you should make it clear that this is a new version of the existing service. Of course, this strategy has the disadvantage that you introduce a new service every time you make a technical change. This strategy should therefore be based on restrict the case that the service has already been put into operation. Non-trivial business versioning A mechanism can be established to prepare services for future enhancements. The infrastructure can offer an extension point where new optional attributes can be added if required. Technical means can be provided to map new service versions to old service versions for enhancements. You can implement an indirection with which different implementations can be used under the same name for different users to be made available. Versioning of data types If services use structured data types, versioning of these data types must also be considered. - Use typed interfaces with different data types. - Use typed interfaces with the same data type. - Use generic interfaces so that the differences do not matter play.
How do you secure the access to SOAP / WSDL Web Services?
Web Services Security (WS-Security) describes enhancements to SOAP messaging to provide quality of protection through message integrity, message confidentiality, and single message authentication. WS-Security also describes how to encode binary security tokens and attach them to SOAP messages. Specifically, the WS-Security profile specifications describe how to encode the following tokens: -Username tokens -X.509 certificates -SAML assertions -Kerberos tickets -LTPA binary tokens
How can transactions be realized with web services? What support does the WS-* Stack offer?
Transactions can be realized with: - Long-term transactions - Nuclear transactions - WS Business Activity and WS Coordination(For long term transactions) - WS Atomic Transaction(will be performed using 2 Phase Commit Protocol (2PC) implemented) WS Business Activity For long term business transactions resources are not lured because transactions run longer. There are compensation activities for errors. WS Coordination Coordinates an activity activity - Activation service - Registration service - Coordination Protocol service Provides frameworks and protocols So that actions of distributed applications can be coordinated WS Protocol Stack provides network protocols for - Define - localization - Implement and - Interacting with Web Services/Services Stack has 4 protocols - (Service) Transport Protocol For transport messages between network applications - (XML) Messaging Protocol For encoding messages - (Service) Description Protocol For the description of Public Interface for Web Services - (Service) Discorver Protocol Centralizes services in a registry so you can see which services are available on the web
How can transactions be realized with RESTful Web Services?
Try-Confirm/Cancel (TCC) Either Cancel Everywhere or Confirm Everywhere Try Confirm/Cancel (TCC) is a business-level protocol for distributed atomic transactions offered by Atomikos. The main difference with this approache is that the transactional events corresponding to cancel ('rollback') and confirm ('commit') are not defined by the needs of the middleware/database but rather by the application/business services. This makes TCC a highly practical and business-oriented protocol, which fits very well within the constraints of the REST uniform interface. The underlying ideas lend to RESTful Web services, without the need to introduce coupling. In fact, applying TCC to REST allows to offer distributed transactions with services that are unaware of being part of such atomic transaction.
How can data be shared across services in a Microservices Architecture?
Via: Messaging Applications exchange messages, typically via a message broker. The broker routes messages to interested parties using topics and queues. Some common systems are RabbitMQ, Kafka, and ActiveMQ. Most systems offer both publish-subscribe and point-to-point message semantics. Sharing Data Stores Another way to share data is to share a data store. Services can share a relational database, NoSQL store, or another data storage service. One or more services publish the data to the database, and other services consume it when required.
