Computing, Programming, and Software Basics
Types of Primatives
Arithmetic Logical Data Movement Control Flow Bit Manipulative Floating Point Synchronization Specialized
What is a remote host?
DEFINITION A computer that resides in some distant location from which data are retrieved; it typically refers to a server in a private network or the public Internet. However, it can also refer to a user's PC in another location that is accessed over the Internet for file transfer or remote control operation. Up until the 1990s, a remote host was almost always a single, centralized computer system that was accessed using terminals directly connected or over private lines or via a dial-up modem.
#programming #coding What is an interpreter?
DEFINITION A python interpreter isiseach high-level program statement into machine code.; it translates the command that you write out into code that the computer can understand. However, to better understand this definition, we must explain what high-level and low-level languages are.
#programming #code what is high v. low-level languages?
DEFINITIONS HIGH Computer programs are generally written in a high-level language, also called source code LOW Machine language
#basics #data What is a Data Structure?
DEFNIITION Made up of a collection of data values, the relationships among them, and the functions or operations that can be applied to the data Focus of Data Structures - efficiency Examples: Arrays: A collection of items stored at contiguous memory locations. Linked Lists: Each item (node) stores a reference to the next item in the list. Trees: A hierarchical structure with nodes, where each node has children and a parent, except the root (which has no parent). Graphs: A collection of nodes connected by edges. Stacks and Queues: Linear structures that follow specific order for operations (LIFO for stacks and FIFO for queues).
#datastructures #basics What is the different between data STRUCTURES and data TYPES?
Data types are more about categorizing individual pieces of data. They define a set of values and the allowable operations on those values. Data structures, are about organizing and storing multiple pieces of data together; they define ways to arrange data sets and allow operations on the entire collection.
#data #datum What is the difference between Data and Datum?
Datum is singular, meaning "one piece of information" or "one numerical result." Data is the plural form of datum and should not be used as a singular noun.
Matrix Operations
Deep learning relies heavily on matrix operations such as matrix multiplication, addition, and transposition. These operations form the backbone of neural network computations, especially in the forward and backward passes of deep learning models.
#programming How does a 1. computational machine work with a 2. web server?
1. ___ ___s takes care of opening and reading files, making database connections, running algorithms and to do all that understanding the specifics and implications of the programming platform. 1. can be a part of 2. 1. ___is consider usually a part or module or engine that computes the response and does all the hard part and communicates back to the 2. __ for the dispatch. 1. __ ___ is more often also a server to coordinate with the front heading web server
Instruction Set Architecture (ISA): Define and tell where it is in the right hardware
Definition: ___ informs the design of both the processor's hardware and the software that runs on it, bridging the gap between the physical operations of the processor and the high-level instructions provided by software applications ___ is implemented within the processor's microarchitecture Where is the microarchitecture? The hardware layer that directly executes the machine code Defines the hardware's operation: Data types, registers, addressing modes, and memory architecture controls which primatives are used, how the machine code must conform - how to "build instructions" for this processor location: part of the processor's microarchitecture and effectively utilizes its microarchitecture for optimal performance
Specialized Primitives
Depending on the processor's design and target applications, there may be specialized primitives for specific tasks, such as graphics processing, encryption and decryption, digital signal processing, and vector operations for scientific computing.
Descriptive Logic means what? And what is the use for this in RAG?
Description Logics: Description logics are a family of formal logic languages specifically designed for representing and reasoning about concepts and their relationships in a structured way. They are commonly used in ontology development and semantic web applications.
APIs, how do they work?
First: the client side sends a request to the server, which first gets validated by the ___ Second: If the request is valid, the ___ then transfers it to the server; the latter creates and sends a response, which again gets validated by the ___, and is then transferred to the client. What Else: ___ is also used to integrate third-party applications and services; allows adding features to the software without having to write your code - for Example: Stripe a. you can add the payment feature to your product b. It's faster and easier because you don't have to deal with payment processing, card data storage, etc.
Floating-Point Primitives
For processors that support floating-point operations, these primitives allow for arithmetic operations (add, subtract, multiply, divide) on floating-point numbers, which are numbers with decimal points.
#programming What is another name for source code?
High level programming languages
Examples of Hosted Hypervisors are?
VMware Workstation VirtualBox Microsoft Hyper-V
Do SDKs typically include APIs?
Yes plus, programming tools, and often documentation and other supporting materials
What is a WEB server?
__ ___ is any source which provides a service on a Client-Server based model, typically over HTTP protocol - The whole and sole task of this thing is to continuously listen to the incoming requests - then understand the specifics of the request as per the knowledge of the protocol(HTTP) - - and dispatch the response back - again with the specifics of the protocol.
What is a remote server? What are the challenges with using the remote server?
__ __s are designed to support users who are not on the local area network (LAN) but need access to it; these __ __sare typically used to store and manage data, run applications, and host websites. CHALLENGES 1. Might have some early issues with configuration: example, double-check firewall settings aren't preventing remote connection 2. You need to get access permissions 2. You need to have the right tools
What is an ISA? What does it do? What does it have to do with compute primatives? Where is it "stored" How does it "work"?
___ and compute primatives stands for instruction set architecture, dicated by the processor WHAT DOES IT DO It defines the machine code (incl. the compute primatives that are allowed) WHERE IS IT STORED Microarchitecture "holds" or "implements" the ISA in the sense that it provides the actual circuitry and mechanisms to execute the instructions defined by the ISA. Here's an analogy: If you consider the ISA as the blueprint of a building, the microarchitecture would be the actual construction material, methods, and design techniques used to bring the blueprint to life. HOW DOES IT WORK Decoding: When software is run on a computer, it's first translated into machine code = defined by the ISA; the CPU fetches these instructions and the microarchitectural components decode them into control signals for various parts of the processor Execution: Different parts of the microarchitecture (like ALUs, FPU, etc.) carry out the operations. How efficiently and quickly these operations are performed is determined by the microarchitecture. Optimizations: Modern microarchitectures have features like out-of-order execution, branch prediction, and pipelining to enhance performance. These are implementation details that are invisible at the ISA level but play a crucial role in how quickly and efficiently the ISA's instructions are executed. Memory Operations: The microarchitecture handles memory operations, including cache access, memory reads/writes, and more. Again, while the ISA might have instructions for loading data from memory, the microarchitecture determines how this is done most effectively.
What are Frameworks? when used?
___ are bits of reusable code that provide specific functionality; they offer a standardized way of building software solutions while saving a great amount of time and serve as a guide to structure code, logic, and other aspects of the backend architecture When used? Backend
Primatives: how do they work? COME BACK TO THIS ONE
___ is one of a set of fundamental language elements that serve as the foundation for a programming language
#compiler Give me the anatomy of a compiler
https://www.cs.man.ac.uk/~pjj/farrell/comp3.html#:~:text=Regardless%20of%20where%20the%20program,next%20stage%20of%20the%20compiler.
What is an executable?
it's a standalone program that you can run directly on your computer without needing to install or configure additional dependencies
What is gRPC? Is it open-source?
#opensource open-source remote procedure call (RPC) system developed by Google HOW IT WORKS it handles the details of serialization (turning structured data into a sequence of bytes) and deserialization (turning that sequence of bytes back into structured data) by using Protocol Buffers
What are the two productivity revolutions for programming?
-creation of higher-level programming languages - ascent of open source v. proprietary software
Static coupling
.how architectural parts (classes, components, services, and so on) are wired together: dependencies, coupling degree, connection points, - an architect can often measure __ ___ at compile time as it represents the static dependencies within the architecture.
Compute Primatives - tell the two contexts of this term, in 1. general and for 2. higher order programming languages
1. In General Computer processors do not understand source code, whether an assembly language or high-level programming language, processors can only understand machine language (binary code consisting of 0s and 1s)* and __ indicate what the processor is capable of doing, such as adding data or loading it into a specific location in memory * and the machine code must conform to the processor's instruction set architecture (ISA), which is part of the processor's microarchitecture; the ISA defines the __ that are supported by the processor, as well as how to build instructions for that processor 2. In High-level Languages Java, etc sometimes discussed in terms of having their own ___. In this case, ___ represent the smallest units of programming logic available to a specific language. The term primitive in this context is usually discussed in relationship to data types. For example, the Java data types int, char and Boolean are all considered "primitive types."
Relational v. Non-relational, what is the difference? Used for what?
1___ use a collection of tables that are related to each other and 2. ___ can store different types of data, which are not interrelated Used For Databases
GPUs and Primatives, Opcodes, and Operands
2. GPUs: - Compute Primitives: GPUs are specialized processors originally designed for computer graphics. They excel at parallel processing and are highly efficient for tasks that require massive parallel computations. The compute primitives of GPUs primarily focus on vectorized operations, such as matrix multiplications, element-wise operations, and image processing operations. GPUs are highly suitable for tasks that can be parallelized, such as deep learning computations. - Opcode and Operand Relationship: GPUs typically use shader programs to execute compute shaders, which are sets of instructions that define the computations to be performed. These shaders contain opcodes specific to the GPU architecture and are written using shader programming languages like CUDA or OpenCL. The operands are the input data that undergoes parallel computations, such as input tensors or matrices. The opcodes direct the GPU to execute the compute primitives on the given operands, taking advantage of parallel processing capabilities.
How do you make a virtual environment in Terminal for Python?
A. Start this way: in terminal in python (conda has "base" like this in the prompt as default- (base) midnighthome : coada create —insertvirtualenvironmentalsocalledtfnamehere tf python=3.11 You have python 3.11.4 so you use that on the example (base) midnighthome: ~ % conda create --pizza tf python=3.11 B. Then create a new project folder: mkdir tf -namegoeshereofproject [example: mkdir tf -pizza] cd tf-namegoeshereofproject [example: cd tf -pizza] C. Setup virtual environment tf -p python3.11.4 env D. Activate it Activate tf
Activation Functions
Activation functions introduce non-linearity into the network, allowing it to learn complex patterns. Common activation functions include ReLU (Rectified Linear Unit), sigmoid, and tanh. GPUs accelerate the computation of these functions across many neurons simultaneously.
CPUs and Primatives, Opcodes, and Operands
CPUs (Central Processing Units), GPUs (Graphics Processing Units), and TPUs (Tensor Processing Units) are different types of processing units optimized for specific types of computations. Each has its own set of compute primitives: 1. CPUs: - Compute Primitives: CPUs have a diverse set of compute primitives. They can perform a wide range of operations, including arithmetic calculations (addition, subtraction, multiplication, division), logical operations, memory operations (loading, storing), and control flow operations. CPUs are designed to handle a variety of tasks efficiently, including general-purpose computing and multitasking. - Opcode and Operand Relationship: CPUs have a complex instruction set architecture (ISA) that includes opcodes representing different instructions. The opcodes define the specific compute primitives to be executed. The operands are the data inputs involved in the computation, such as values stored in memory, registers, or input/output devices. The opcodes direct the CPU to perform specific compute primitives on the given operands.
Control Flow Primitives
Control flow operations influence the execution sequence of instructions in a program. Examples include jumps (unconditional branch), conditional branches (e.g., jump if equal, jump if less than), and subroutine calls and returns.
Convolution Operations
Convolutional Neural Networks (CNNs), widely used for image and video recognition tasks, rely on convolution operations to process input data. Efficiently implementing convolution operations is key to high-performance deep learning models.
#datastructures #basics What is a dynamic data structure? Give an example of one
DEFINITION AKA as dynamic structures can change in size during the execution of a program. Unlike static data structures, which have a fixed size after their creation, dynamic structures can grow or shrink as needed. Key Points: Memory Efficiency: Only the memory required at a particular instant is allocated, avoiding wastage. Flexibility: Suitable for scenarios where the exact amount of data is unpredictable. Overhead: Managing the dynamic allocation and deallocation can introduce overhead in terms of processing time. EXAMPLE Binary Search Tree: A tree data structure where each node has at most two children, and the left child is less than the parent, and the right child is greater. Nodes can be added or removed, allowing the structure to change in size.
#google #algorithms #mustknow What is Breadth-First Search (BFS)?
DEFINITION An algorithm for traversing or searching tree or graph data structures. HOW DOES IT WORK 1. It starts at the tree root and 2. Then explores all of the neighbor nodes at the present depth 3. before moving on to nodes at the next depth level. EXAMPLE from collections import defaultdict, deque class Graph: def __init__(self): self.graph = defaultdict(list) def addEdge(self, u, v): self.graph[u].append(v) def BFS(self, start): visited = set() # To keep track of visited nodes queue = deque() # Use a queue to implement BFS queue.append(start) visited.add(start) while queue: vertex = queue.popleft() print(vertex, end=" ") for neighbour in self.graph[vertex]: if neighbour not in visited: queue.append(neighbour) visited.add(neighbour) # Driver code to test the BFS function g = Graph() g.addEdge(0, 1) g.addEdge(0, 2) g.addEdge(1, 2) g.addEdge(2, 0) g.addEdge(2, 3) g.addEdge(3, 3) print("Breadth First Traversal starting from vertex 2:") g.BFS(2) OUTPUT https://colab.research.google.com/drive/1cjriJRb_u7fRlzwEjbl7gpQ-YRy3Tg3z#scrollTo=T2g4_1Lo1jyU Start BFS from node 2. The neighbors of 2 are 0 and 3. Visit 0 first (based on the order they appear in the adjacency list). Next, visit all neighbors of 0 which are 1 and 2. Since 2 is already visited, we move on to 1. Finally, visit all neighbors of 1. Only neighbor is 2 which is already visited. The BFS traversal order is 2 -> 0 -> 3 -> 1.
#google #algorithms What is a Binary Search Tree (BST)?
DEFINITION ___ is a data structure where every node has at most two children (left and right) and satisfies the two conditions: 1. all nodes to the left are less than the current node 2. and all nodes to the right are greater than the current node EXAMPLE #create a simple Binary Search Tree (BST) with the following data: 50, 30, 70, 20, 40, 60, 80. class Node: def __init__(self, key): self.left = None self.right = None self.val = key def insert(root, key): # If the tree is empty, return a new node if root is None: return Node(key) # If the key is smaller than the root's key, insert into the left subtree if key < root.val: root.left = insert(root.left, key) # If the key is larger than the root's key, insert into the right subtree else: root.right = insert(root.right, key) return root def inorder_traversal(root): """A utility function to do inorder tree traversal""" return inorder_traversal(root.left) + [root.val] + inorder_traversal(root.right) if root else [] # Driver program to test the above functions r = Node(50) r = insert(r, 30) r = insert(r, 20) r = insert(r, 40) r = insert(r, 70) r = insert(r, 60) r = insert(r, 80) # Print in order traversal of the BST print(inorder_traversal(r)) OUTPUT: https://colab.research.google.com/drive/1xetRWaIZs1w27diaClZn_uZHGFdB0G4x#scrollTo=4L0A7K6orzF8 In this tree: 50 is the root. 30 and 70 are children of 50. 20 and 40 are children of 30. 60 and 80 are children of 70.
#basics #data What is a Data Type?
DEFINITION specifies what kind of information that can be stored in a specific location, and specify how we can operate on that data EXAMPLES integer, floating-point, Boolean, etc.
#basics #data What is a Data Value?
DEFINITION the actual piece of information or content that is stored somewhere (in a variable, in a database, etc.) - the specific content or datum that a program will act upon or manipulate
#testing #programming Finish this sentence: A unit test is a _____
DEFINITION type of software testing where individual units or components of a software are tested. The primary purpose is to validate that each unit of the software code performs as expected. Here are some key points about unit tests: Granularity: Unit tests typically test functions, methods, or classes in isolation. - This means that when you're running a unit test, you're testing the smallest piece of executable software code (like a function or method) to ensure that it behaves correctly in all scenarios you've defined. Isolation: One of the main tenets of unit testing is isolating the piece of code being tested from the rest of the software system, including databases, filesystems, network connections, and other external processes. This often involves using "mocks" or "stubs" to simulate external systems. Automated: Unit tests are usually automated, meaning once they're written, they can be run automatically by software without human intervention. This makes it easy to run tests frequently and consistently. Quick Feedback: Because they're automated and have a small scope, unit tests can provide quick feedback to developers. If a unit test fails, it indicates that a recent change in the codebase has introduced an error. Documentation: A well-written set of unit tests can serve as documentation. They show how a piece of software is expected to behave and how to use it. Development Paradigms: There are development methodologies like Test-Driven Development (TDD) where writing unit tests is integral. In TDD, developers write tests before they write the actual code. The idea is to ensure that every piece of functionality is under test. Tools: Many tools exist to help write and run unit tests in almost every major programming language. Examples include JUnit (for Java), pytest (for Python), and Mocha (for JavaScript). In essence, unit tests help to ensure that individual parts of a codebase work as intended, making it easier to catch and fix bugs early in the development process. They are a foundational aspect of software quality assurance and are vital for maintaining the long-term health and reliability of software applications.
#google #mustknow What is a merge sort algo? How does it work? When do you use it?
DEFINITION A divide-and-conquer sorting algorithm that divides the list into n sub-lists, each containing one element, and then repeatedly merges sub-lists to produce sorted ones. HOW IT WORKS 1. first, recursively divides the array into halves 2. second, we get subarrays with a size of 1 3. then, we merge these subarrays (which are inherently sorted) to produce larger, sorted subarrays 4. Until the entire array is sorted WHEN TO USE data engineering: data preprocessing and analysis EXAMPLE
#programming What is the difference between 1. interpreters and 2. compilers?
DEFINITIONS 1. ___ translate source code one statement at a time; because of this, they don't create any object code and so are memory efficient (but slower), and you can debug your code line by line 2. ___ first scans the entire program and then translates the whole program into machine code, but overall can run your code faster
1. Opcodes, 2. Operands, and 3. Primatives: Define 1 and 2, and how they work with 3.
DEFINITIONS 1.___s are specific to the processor architecture and are typically accompanied by 2.___s which represent the data to be manipulated by the 3.___ 2.___s are the binary equivalent of the 3. ___ - and tells the processor what operation to carry out How They Work Together: An add primitive might translate to an operand such as 0011, 0100, 0110 or another set of bits (depending on the underlying architecture)
#basics #mustknows What is dynamic programming and why is it critical to know this concept? How is this related to generative AI?
Dynamic programming (DP) is a computer programming technique that breaks down a problem into smaller sub-problems. The results of each sub-problem are saved so they don't need to be computed again. The sub-problems are then optimized to find the overall solution. EXAMPLES IN GEN AI is related to sequence generation, like Beam Search, a heuristic search algorithm used in machine translation and speech recognition. Instead of producing one word/token at a time, it keeps multiple sequences (called "beams") at each step, expanding all of them, and then selecting the top few based on their probabilities. The "expanding and selecting" process is akin to dynamic programming, where decisions are made based on previously computed values (probabilities of sequence prefixes in this case). IMPORTANCE It's a foundation in almost all programs, including basic algorithms, simple variables, and complex data structures. HISTORY Dynamic programming was invented by American mathematician Richard Bellman in the 1950s. The goal of dynamic programming is to create a solution that preserves previously seen values to increase time efficiency. EXAMPLES USED IN INTERVIEWING The knapsack problem Finding the maximum and minimum range of an algorithmic query Solving Ramsey's problem of optimal saving APPLICATIONS WHICH USE DYNAMIC PROGRAMMING Finding the shortest path between any two nodes in a weighted graph Seam carving Machine learning and genomics Cryptography
#tokenizer Yes, programming uses the word "tokenizer" but it means something related to the compiler, what is it?
Every program must first pass through a Tokenizer, or as it is sometimes called, a Lexer DEFINITION responsible for dividing the input stream into individual tokens, identifying the token type, and passing tokens one at a time to the next stage of the compiler
First Order Logic is what?
FOL is a formal logic language that uses variables, predicates, quantifiers, and logical connectives to represent statements and relationships between objects. It is widely used for knowledge representation and automated reasoning.
Synchronization Primitives
In multi-threaded or multi-core processors, synchronization primitives help manage the coordination between different threads or processes. These include operations for locking, unlocking, and atomic operations that ensure a sequence of operations is executed without interruption.
Structured Knowledge means what?
Knowledge expressed in formal logic languages, such as first-order logic or description logics, with well-defined syntax and semantics for reasoning.
Logical Primitives
Logical operations are used to perform bit-level manipulation and comparisons. Common logical primitives include AND, OR, NOT, XOR (exclusive OR), and comparison operations like equal to, less than, and greater than.
#deeplearning Which types of primitives become particularly important due to the nature of deep learning algorithms and the parallel processing capabilities of GPUs.
Matrix Operations
Reduction Operations
Operations like sum, mean, and max are used for aggregating data, such as in the pooling layers of CNNs or when computing loss functions. GPUs can perform these reductions quickly over large datasets.
Dynamic coupling
Refers to how architecture parts call one another, what kind of communication, what information is passed, strictness of contracts, etc.
#remove What is SSH?
Secure Shell DEFINITION a network protocol that connects users to a remote computer over a secure connection; allows administrators and other authorized users to connect to secure computers over a network that is not secure, like the Internet. This is accomplished through the use of encryption.
SSH
Secure Shell - the acronym stands for Secure SHell
SDK stands for what?
Software Development Kit:
TPUs and Primatives, Opcodes, and Operands
TPUs are specialized hardware accelerators developed by Google specifically for deep learning workloads. They are designed to perform large-scale matrix calculations efficiently. TPUs focus on tensor operations and deliver exceptional performance for neural network computations. They feature a significant number of multiply-accumulate (MAC) units, which are utilized in deep learning tasks. - Opcode and Operand Relationship: TPUs utilize a domain-specific language called TensorFlow Processing Units (TPU MLIR) to describe computations. These descriptions are then compiled into executable opcodes optimized for the TPU architecture. The operands are tensors, which are n-dimensional data structures, used as inputs and intermediate data during the computations. The opcodes direct the TPU to execute specific tensor operations, like matrix multiplications or convolutions, taking advantage of the hardware's specialization in deep learning tasks
Primatives in the context of the Higher Order Programming Languages
The term primitive might also be used in other ways in a high-level programming language. For example, it might refer to operators, expressions, procedures or other language elements. Regardless of how it's used, a primitive in a high-level language does not necessarily have a direct counterpart in machine language. For example, adding two numbers in a high-level language might represent a single primitive operation in that language, but when translated to machine code, that operation will likely require multiple lower-level primitive operations.
back end logic, define?
The way that the entities are modeled and the rules applied to those entities that all together reflect the business requirements that was created to run almost exclusively on servers, interpreting inputs and producing outputs.
Element-wise Operations
These include basic arithmetic operations (addition, subtraction, multiplication, division) performed element-wise on matrices or tensors. Element-wise operations are ubiquitous in deep learning for operations such as adjusting weights during backpropagation. Random Number Generation: Stochastic aspects of deep learning, such as initializing weights or dropout (a regularization technique), require the generation of random numbers. GPUs offer fast random number generation primitives. Synchronization Primitives: Given that GPUs often run multiple threads in parallel, synchronization primitives are necessary to ensure that data dependencies are respected across different parts of a deep learning algorithm. Memory Management Primitives: Efficiently managing memory accesses and transfers between the CPU and GPU memory spaces is crucial for high-performance deep learning applications. This includes primitives for memory allocation, deallocation, and data movement.
Bit Manipulation Primitives
These operations allow for direct manipulation of individual bits within a data word. They include shifting (shift left, shift right), rotation, and setting or clearing specific bits.
Arithmetic Primitives
These operations involve basic arithmetic calculations, such as addition, subtraction, multiplication, and division. They can operate on various data types, including integers and floating-point numbers.
Data Movement Primitives
These primitives are responsible for moving data between different parts of a computer system, such as loading data from memory into a register (LOAD), storing data from a register into memory (STORE), and moving data between registers (MOVE).
What is code refactoring? Why do it?
___ is the process of restructuring existing computer code—changing the factoring—without changing its external behavior - with the intention of improving the design, structure, and/or implementation of the software (its non-functional attributes), while preserving its functionality WHEN TO DO - Every time you can WHAT "HAPPENS" it most often comprises applying a series of standardized, basic actions, sometimes known as micro-refactorings. IDEs provide automated support for performing the mechanical aspects of these basic refactorings. EXPECTED RESULTS - Debugging: done well, code refactoring may help software developers discover and fix hidden or dormant bugs or vulnerabilities in the system by simplifying the underlying logic and eliminating unnecessary levels of complexity. - Improved Performance: programs perform faster or use less memory
soft update is what?
___ refer to when the user is shown a prompt that a new version is available and they can update their app to the new version if they want to.
What is a "run configuration"?
____ s a set of parameters, arguments, and settings that define how an application is launched.
hard update is what?
___refer to when the user is forced to update the client version to a higher version number than what is installed on their mobile.
What do SDKs provide?
a set of software tools and programs provided by hardware and software vendors that developers can use to build applications for specific platforms
#mustknow #deeplearning What is a computational graph?
not JUST for Deep Learning DEFINITION a directed graph where nodes represent operations or variables and edges represent the flow of data between these operations or variables WHY IN USE The graph provides a structured way of representing mathematical expressions and computations, and it offers a clear visualization of how data moves through various operations. HOW IT WORKS Nodes: These represent operations (like addition, multiplication) or variables (like input data, weights, biases). Edges: These represent tensors (multi-dimensional data arrays) that flow between nodes. USE CASES, DEEP LEARNING Automatic Differentiation: By having the entire computation represented as a graph, it becomes straightforward to compute gradients using techniques like backpropagation. This is particularly useful in training neural networks, where gradients are needed to update weights. Parallelism: Computational graphs can highlight operations that are independent of each other and can be executed in parallel, allowing for more efficient computation, especially on hardware like GPUs. Optimization: Some frameworks use the computational graph structure to optimize operations, like combining consecutive operations into a single operation, which can lead to faster execution. Visualization: They offer a clear way to visualize complex operations, especially in deep neural networks. USE CASES, OUTSIDE OF DEEP LEARNING General Machine Learning: Before the rise of deep learning, computational graphs were used in other machine learning frameworks to represent and evaluate models. Automatic Differentiation: Computational graphs provide a mechanism for automatic differentiation, which is a technique used to compute derivatives of functions. This capability isn't limited to deep learning; any application or field that requires optimization of functions can benefit from it, such as physics simulations or financial modeling. Symbolic Mathematics: Systems like Mathematica or SymPy use computational (or expression) graphs to represent mathematical expressions symbolically and then manipulate, simplify, or evaluate these expressions. Graphical Models in Probabilistic Reasoning: Bayesian networks and factor graphs are graphical model
#setup What is a typical programming file set up to use every time you program/ no matter what programming language? Start with "Project Name"
project_name/ ├── src/ ├── tests/ ├── setup.py ├── README.md ├── requirements.txt └── .gitignore
#basics #data How do you put together data types and data strucutres?
you can think of data types as the building blocks, and data structures as the structures or systems you build using those blocks. EXAMPLES 1. array (data structure) can hold multiple integers (data type) 2. a tree (data structure) can be made up of nodes that store strings (data type)
Hypervizor
—- also known as a virtual machine monitor, is a software layer that enables the creation and management of virtual machines (VMs) on a physical computer or server. The primary function of a hypervisor is to divide the physical resources of a computer, such as processing power, memory, and storage, into multiple virtual environments known as virtual machines. Each virtual machine behaves like an independent computer system with its own operating system, applications, and resources. The hypervisor controls and allocates the physical resources among the virtual machines, ensuring isolation and security between them. It also manages the communication between the VMs and the underlying hardware, allowing them to run simultaneously on the same physical machine. There are two types of hypervisors: 1. Type 1 or Bare Metal Hypervisor: This type of hypervisor is installed directly on the physical hardware of a server or computer. It runs directly on the underlying system without the need for an operating system, providing direct control and efficient resource utilization. 2. Type 2 or Hosted Hypervisor: This type of hypervisor is installed on top of an operating system, like any other application. It relies on the underlying operating system for hardware access and resource management. Both types of hypervisors offer the flexibility to create and manage multiple virtual machines, providing benefits such as hardware consolidation, resource utilization, and the ability to run multiple operating systems or applications on a single physical machine.
