Chap 5 PART 1
A page table keeps track of where put in the page our physical memory PA
(PPN,offset)
VA = 32 bits
(size of address space 2 32=4 GB)
Hit time is the time to access the upper level of the memory hierarchy
, which includes the time needed to determine whether the access is a hit or a miss (that is, the time needed to look through the books on the desk). hit time will be much smaller than the time to access the next level in the hierarchy
Moreover, if it is, how do we find it The simplest way to assign a location in the cache for each word in memory is to assign the cache location based on the address of the word in memory.----DIRECT MAPPING Only have one choice:
- (Block address) modulo (# of Blocks in cache) Number of blocks is a power of 2 Because there are eight words in the cache, an address X maps to the direct-mapped cache word X modulo 8. Th at is, the low-order log2 (8) 3 bits are used as the cache index.
- 1 GB memory ➔ 1 GB = 2 30 bytes ➔ PA 30 bits
- 4 GB memory ➔ 4 GB = 2 32 bytes ➔ PA 32 bits - 512 MB memory ➔ 512 MB = 2 29 bytes ➔ PA 29 bits
Average memory access time (AMAT) - AMAT = Hit time + Miss rate × Miss penalty • Example - CPU with 1ns clock, hit time = 1 cycle, miss penalty = 20 cycles, I-cache miss rate = 5%
- AMAT = 1 + 0.05 × 20 = 2ns • 2 cycles per instruction
Ideal memory
- Access time of SRAM - Capacity and cost/GB of disk
Primary cache
- Focus on minimal hit time (direct map minimizes hit time) - Also we separate I-cache and D-cache. - Data path accesses memory at least once (IF) and maybe twice (MEM) for each instruction. To avoid structural hazards, we split the L1 cache for instruction and data memory
Try to minimize page fault rate
- Fully associative placement
Spatial locality
- Items with memory addresses near those accessed recently are likely to be accessed soon - E.g., sequential instruction access, sequential access to array data . Libraries put books on the same topic together on the same shelves to increase spatial locality. Space in array for ex. Access i will bring i and i+1, i+2
Larger blocks fewer of them • But in a fixed-sized cache • More competition increased miss rate
- Larger blocks pollution • Larger miss penalty - Can override benefit of reduced miss rate - Early restart and critical-word-first can help
Page size 1 KB: 1KB = 2 10 bytes ➔ page offset 10 bits
- Page size 4 KB: 4KB = 2 12 bytes ➔ page offset 12 bits
- Virtual address space divided into pages
- Physical memory (RAM)divided into page frames - Each page can be brought to a page frame and will fit perfectly
The Principle of Locality:
- Programs access a relatively small portion of the address space at any instant of time
Access to a sector involves
- Queuing delay if other accesses are pending - Seek: move the R/W heads mechanically (long delay) - Rotational latency (mechanical delay — long) The time required for the desired sector of a disk to rotate under the read/write head; usually assumed to be half the rotation time. - Data transfer - Controller overhead(access data/make it available to the rest( Lesson: Once you locate your data, read as many bytes as you can! (don't read individ. Bytes from disks but actual blocks)
Each sector records
- Sector ID - Data (512 bytes, 4096 bytes proposed) - Error correcting code (ECC) • Used to hide defects and recording errors
On page fault, the page must be fetched from disk
- Takes millions of clock cycles - Handled by OS code
What if there is no data in a location?
- Valid bit: 1 = present, 0 = not present - Initially 0
VA(VPN, OFfset) find number of bits VPN? virtual address=16 bits offset bits=12 bits
16-12=4 bits VPN
how many entries/size of page table number
2^VPN
Fully Associative A cache structure in which a block can be placed in any location in the cache all the entries in the cache must be searched because a block can be placed in any one. To make the search practical, it is done in parallel with a comparator associated with each cache entry.
8 way set associative-> 8 blocks/set. 8/8 is 1 set
Alternate of write through: write back
: On data-write hit, just update the block in cache Keep track of whether each block is dirty (i.e., modified) • When a dirty block is replaced Write it back to memory Can use a write buffer to allow replacing block to be read first Write-back schemes can improve performance,
WRITE BACK AND WRITE THROUGH ASSUME
A WRITE HIT
A tag fi eld, which is used to compare with the value of the tag fi eld of the cache ■
A cache index, which is used to select the block Cache blocks are 1 word/4 bytes. Data stored in cache has 4 bytes.
How freq can you initiate an access?
A little kid can ask money from Dad on Sun, Mom on Monday, Grandma on Tues.... All family members are like memory banks. These memory banks have longer cycle times, initiate access quicker. Memory banks are great overlapping access time with memory time.
Alternatives for write-through
Allocate on miss: fetch the block and write Write around: don't fetch the block and • Since programs often write a whole block before reading it (e.g., initialization) • For write-back - Usually fetch the block
NAND
Bit cell like a NAND gate Denser (bits/area) but block at time Cheaper /GB
NOR Flash
Bit cell like a NOR gate Random read/write access Use for instruction memory
Mapping an Address to a Multiword Cache Block Consider a cache with 64 blocks and a block size of 16 bytes. To what block number does byte address 1200 map?
Block address=byte add/bytes per block 1200/16 bytes per block=block #75 Block number 75 module 64=11. This is block in cache index in cache to which block 75 maps
Sizes typically in powers of 2.
Can use higher order address bits as page numbers, and lower address bits as offsets within page.
Take Advantage of Locality Memory hierarchy Store everything on disk
Copy recently accessed and disk-> smaller Copy more recently accessed (and nearby) items from DRAM to smaller SRAM memory - Cache memory attached to CPU Processor: Cache(faster)--main(slower) Cache memory: level of memory hierarchy closest to CPU
middle in size, speed, and price
DRAM
DRAM (Read/Write) Cycle Time [much longer]>>
DRAM (Read/Write) Access Time
3 APPROACHES TO MAPPING
Direct Mapping Cache--most rigid/only one location Set Associative Cache Fully Associative Cache--least rigid/most flexible More flexible, decrease miss rate, and increase hit rate
Increasing the associativity increases the number of blocks per set
Each increase by a factor of 2 in associativity doubles the number of blocks per set and halves the number of sets
Each disk surface is divided into concentric circles, called tracks. There are typically tens of thousands of tracks per surface.
Each track is in turn divided into sectors that contain the information
to actually find VPN
FLOOR VA/PAGE SIZE Physical frame number = 6 • Physical address = 6 ∗ 4096 + 4 • = 24576 + 4 = 24580
Magnetic disks are nonvolatile like fl ash, but unlike fl ash there is no write wear-out problem.
However, fl ash is much more rugged and hence a better match to the jostling inherent in personal mobile devices.
How do we know if a data item is in the cache?
If each word can go in exactly one place in the cache, then it is straightforward to find the word if it is in the cache
This second-level cache is normally on the same chip and is accessed whenever a miss occurs in the primary cache. If the second-level cache contains the desired data, the miss penalty for the fi rst-level cache will be essentially the access time of the second-level cache, which will be much less than the access time of main memory .
If neither the primary nor the secondary cache contains the data, a main memory access is required, and a larger miss penalty is incurred.
all tags of all the blocks in the cache must be searched. Associativity increases as long as you keep the size of cache the same.
If num of blocks within the same, associativity increases number of sets The advantage of increasing the degree of associativity is that it usually decreases the miss rate,
miss rate
If the data is not found in the upper level, the request is called a miss. The lower level in the hierarchy is then accessed to retrieve the block containing the requested data. The miss rate (1−hit rate) is the fraction of memory accesses not found in the upper level.
HIT
If the data requested by the processor appears in some block in the upper level, this is called a hit (analogous to your finding the information in one of the books on your desk).
Position and function of the MMU (Memory Management Unit) CPU has a virtual address that is fed to the MMU (uses TLB and page tables(TLC and page tables updating done by OS) to generate a physical address). Out comes physical address and sents to memory. If that page is in memory, data is sent back. If not, page fault. Go to disk controller
In page fault, OS comes and swaps out processes that cause the page fault and gives CPU to another process . IT can't wait
If tag bits do not match:miss
Index bit to cache memory: CHECK valid bit. If one, then check tag bits. Yes it is hit or not miss in cache
Temporal locality
Items accessed recently are likely to be accessed again soon -= e.g., instructions in a loop, induction variables In a loop, you access one instruction into memory, put in cache, run a loop again, and get in cache. If you recently brought a book to your desk to look at, you will probably need to look at it again soon.
L-2 cache - Focus on low miss rate to avoid main memory access (more associativity) - Hit time has less overall impact because L2 cache accessed less frequently. - Unified
L-1 cache usually smaller than a single cache - L-1 block size smaller than L-2 block size
Why have larger blocks?
Larger blocks should reduce miss rate - Due to spatial locality • But in a fixed-sized cache - Larger blocks fewer of them • More competition increased miss rate - Larger blocks pollution • Larger miss penalty - Can override benefit of reduced miss rate - Early restart and critical-word-first can help
Dynamic Ram
Larger capacity, slower, cheaper In a dynamic RAM (DRAM), the value kept in a cell is stored as a charge in a capacitor. A single transistor is then used to access this stored charge, either to read the value or to overwrite the charge stored there.
Memory Hierarchy
Level 1:closer to processor Multiple levels of cache: SRAM Main memory(DRAM) Secondary Memory(disks) A typical memory hierarchy take advantage of principle of locality. Can present user with as much memory as is available in cheapest technology
Given - I-cache miss rate = 2% - D-cache miss rate = 4% - Miss penalty = 100 cycles - Base CPI (ideal cache) = 2 - Load & stores are 36% of instructions
Miss cycles per instruction - I-cache: 0.02 × 100 = 2 - D-cache: 0.36 × 0.04 × 100 = 1.44 Actual CPI = 2 + 2 + 1.44 = 5.44 - Ideal CPU is 5.44/2 =2.72 times faster
Cache Misses The control unit must detect a miss and process the miss by fetching the requested data from memory
Modifying the control of a processor to handle a hit is trivial; misses, however, require some extra work. On cache hit, CPU proceeds normally
fully associative cache Slower hit determ time/ compare multiple locations
Multiple locations in memory will map to the same block in cache. Mod 8 <- memory add takes lower 3 bits (2^3=8)
FLASH MEMORY: With wear leveling, personal mobile devices are very unlikely to exceed the write limits in the flash. Such wear leveling lowers the potential performance of flash, Not suitable for direct RAM/direct replacement
Nonvolatile, semiconductor storage tend to be faster to access than magn disks Smaller in capacity, lower power, more robust
Increase bandwidth by Access data from memory, OR organize memory into banks
OR organize memory into banks Processor can access words arranged on factory banks. Access whichever memory banks as data and bring to processor. Once accessed, you have to wait until access it
E.g 32 bit address cache with 1024 blocks. Each block now is 16 bytes.
Offset det by how many power of 2 bytes size is? 4 Index det by how many powers of 2 num of blocks? 10 bits Tag is remaining of 32 bits TAG INDEX OFFSET 18 10 4
E.g 32 bit address cache of 512 blocks. Each block is 64 bytes.
Offset det by how many power of 2 bytes size is? 6 Index det by how many powers of 2 num of blocks? 9 Tag is remaining of 32 bits TAG INDEX OFFSET 17 9 6
E.g 1024 blocks each with 4 byte.
Offset det by how many powers of 2 bytes size is? 2. Index det by how many powers of 2 num of blocks? 10 bits Tag is remaining of 32 bits TAG INDEX OFFSET 20 10 2
page size 1024 bytes (1KB/2^10 bytes) • Address = 1300 • Virtual page number = [1300/1024] = 1 remaining 22 bits of page • Offset = 1300 mod 1024 = 276. It will be the lower 10 bits
Offsets are the same in both virtual and physical address space
To access data, the operating system must direct the disk through a three-stage process. Th e fi rst step is to position the head over the proper track. Th is operation is called a seek, and the time to move the head to the desired track is called the seek time.
Once the head has reached the correct track, we must wait for the desired sector to rotate under the read/write head. Th is time is called the rotational latency or rotational delay. Th e last component of a disk access, transfer time, is the time to transfer a block of bits. Th e transfer time is a function of the sector size, the rotation speed, and the recording density of a track
Increased Bandwidth
One-word-wide memory organization is slow
The disk heads for each surface are connected together and move in conjunction, so that every head is over the same track of every surface.
PACK more bits/area. Way to store is bit of info/change polarity of magnetic field flux on that material
• If page is not present
PTE can refer to location in swap space on disk
translation using page table
Page table in memory in OS kernel address space Page table base register is loaded with the page table base address OS maintained and updates all this when CPU is processed from one process to another
Page Tables Stores placement information (1 PT/process) Array of page table entries, indexed by virtual page number
Page table register in CPU points to page table in physical memory
replacement polci random as well
Random - Gives approximately the same performance as LRU for high associativity - non-usage-based As associativity increases, implementing LRU gets harder
fastest, smallest, most expensive memory
SRAM
Static Ram
Small, faster, more money, more area less dense Used for cache
Magnetic disk
Solid state drivers Much cheaper, larger capacity, and much slower
write through solution
Solution: write buffer - Holds data waiting to be written to memory. stores the data while it is waiting to be written to memory. - CPU continues immediately • Only stalls on write if write buffer is already full. until there is an empty position in the write buffer,
block (or line) .
Th e minimum unit of information that can be either present or not present in a cache. It is basically one book in our example
In a fully associative cache, there is eff ectively only one set, and all the blocks must be checked in parallel.
Th us, there is no index, and the entire address, excluding the block off set, is compared against the tag of every block. In other words, we search the entire cache without any indexing.
Write Miss (data is not in cache) What should happen on a write miss Write Allocate:
The block is fetched from memory and then the appropriate portion of the block is overwritten An alternative strategy is to update the portion of the block in memory but not put it in the cache, called no write allocate.
magnetic disk" The metal platters are covered with magnetic recording material on both sides, similar to the material found on a cassette or videotape. To read and write information on a hard disk, a movable arm containing a small electromagnetic coil called a read-write head is located just above each surface.
The entire drive is permanently sealed to control the environment inside the drive, which, in turn, allows the disk heads to be much closer to the drive surface.
DRAMs buffer rows for repeated access. The buffer acts like an SRAM; by changing the address, random bits can be accessed in the buffer until the next row access.
This capability improves the access time significantly, since the access time to bits in the row is much lower
Four way Set Associative
Total blocks 8. How many sets 8/4=2. 2 sets with 4 blocks each
Set Associative (2 Way Set Associative) Maps to only one set/has a fixed number of locations (at least two) where each block can be placed. Can be in any location within set
Total blocks:8. How many sets? 8/2=4. 4 sets with each 2 blocks In a set-associative cache, the set containing a memory block is given by (Block number) modulo (Number of sets in the cache) all the tags of all the elements of the set must be searched.
Virtual Address=
VPN, OFFSET
What happens to a write miss?
We first fetch the words of the block from memory. After the block is fetched and placed into the cache, we can overwrite the word that caused the miss into the cache block. We also write the word to main memory using the full address
Write-Through On data-write hit, could just update the block in cache - But then cache and memory would be inconsistent
Write through: update cache & also update memory--update both the cache and the next lower level of the memory hierarchy, ensuring that data is always consistent between the two. • But makes writes take longer
The miss penalty is the time to replace
a block in the upper level with the corresponding block from the lower level, plus the time to deliver this block to the processor (or the time to get another book from the shelves and place it on the desk).
If neither the primary nor the secondary cache contains the data,
a main memory access is required, and a larger miss penalty is incurred.
On cache miss - Stall the CPU pipeline - Fetch block from next level of hierarchy - Instruction cache miss • Restart instruction fetch
a. If an instruction access results in a miss, then the content of the Instruction register is invalid. b. Send the original PC value (current PC - 4) to the memory. c .Instruct main memory to perform a read and wait for the memory to complete its access. d. Write the cache entry, putting the data from memory in the data portion of the entry, writing the upper bits of the address (from the ALU) into the tag field, and turning the valid bit on. e.Restart the instruction execution at the fi rst step, which will refetch the instruction, this time fi nding it in the cache.
Cache memory(managed in hardware) .
acts as cache for main memory Virtual memory is for data on secondary storage Data moves slow/large capacity. Read:secondary memory, main, second level Write: opp direction
how else to improve cache perfomance
add associativity increase hit rate reduce miss rate hitr+missr=1
Because each cache location can contain the contents of a number of different memory locations, how do we know whether the data in the cache corresponds to a requested word? (how do we know whether a requested word is in the cache or not? )
adding a set of tags to the cache. The tags contain the address information required to identify whether a word in the cache corresponds to the requested word.
CPU and OS translate virtual
addresses to physical addresses during execution time
Data is copied between only two
adjacent levels at a time The upper level—the one closer to the processor—is smaller and faster than the lower level, since the upper level uses technology that is more expensive.
fully associative cache
allow a given block to go in any cache requires all entries to be searched at once tag comparator per entry
As DRAMs store the charge on a capacitor, it cannot be kept indefinitely
and must periodically be refreshed. To refresh the cell, we merely read its contents and write it back
Translating used the virtual page number
and uses it as an index into this page table
You break the cache into a number of sets. Within each set, n blocks into which data can be placed anywhere. Direct mapped->
block address in memory to a single location in the upper level of the hierarchy. Maps to only one location
VM managed jointly
by CPU hardware and operating system
virtual memory main memory is
cache for secondary storage(disk)
VM translation of a miss is called a page fault. If page is not in memory,
cached/sitting on disk and needs to be brought from main memory(disk). Page not in memory and not in cache
faster memory is
close to the processor and the slower, less expensive memory is below it.
flash memory: To cope with such limits, most fl ash products include a
controller to spread the writes by remapping blocks that have been written many times to less trodden blocks. Th is technique is called wear leveling.
Programs share physical main memory(RAM)
each gets a private virtual address space holdings its freq used code and data
n-way set associative
each set contains n entries block number determines which set search all entries in a given set at once
Size of the page table is determined by how many bits the virtual page number
has. 20 bits for the virtual page number then the page table has 2^20 entries
Making the chip wider also
improves the memory bandwidth of the chip.
To avoid performance loss, the bandwidth of main memory is
increased to transfer cache blocks more efficiently
024 blocks/sets of 4 bytes total cache size in 4 way associativity 1024/4=256 sets NOW THESE ARE THE SETS Offset: how many powers of 2 for bytes/ cache size=2 Index: how many powers of 2 for set number (256)=8 Tag is 22 bits
index: how many powers of 2 for set number
- Data cache miss • Complete data access Th e control of the cache on a data access
is essentially identical: on a miss, we simply stall the processor until the memory responds with the data
With eight blocks, an eight-way set associative cache
is the same as a fully associative cache.
If the miss penalty increased linearly with the block size,
larger blocks could easily lead to lower performance. To avoid performance loss, the bandwidth of main memory is increased to transfer cache blocks more efficiently
slowest speed , biggest, and cheapest
magnetic disk
increase associativity decreases
miss rate
The use of a larger block decreases the
miss rate and improves the efficiency of the cache by reducing the amount of tag storage relative to the amount of data storage in the cache.
The faster memories are
more expensive per bit than the slower memories and thus are smaller.
memory can complete writes < the rate at which the processor is generating writes,
no amount of buffering can help, because writes are being generated faster than the memory system can accept them.
• If page is present in memory PTE stores the physical page frame
number Plus other status bits (referenced, modified, ...)
Associativity increases and number
of index bits decrease, tag bits increase
The hit rate, or hit ratio, is the fraction
of memory accesses found in the upper level; it is often used as a measure of the performance of the memory hierarchy
The principle of locality states that programs access a relatively small portion .
of their address space at any instant of time, just as you accessed a very small portion of the library's collection
VM Black is called a page:
page size is a power of 2
virtual address is translated to
physical address Bring pages from address space of the processes which are private to disk and bring them into frames (page frames) Protected from other programs'
Increase bandwidth by Access data from memory,
put in cache that is parallel, read multiple words and put in cache. Processor will access that one at a time, but much faster sitting in cache
static RAM used for
simply integrated circuits that are memory arrays with (usually) a single access port that can provide either a read or a write.
as the distance from the processor increases, the
size of the memories and the access time both increase
SRAM have a fixed access time to any datum don't need to refresh and
so the access time is very close to the cycle time. SRAMs typically use six to eight transistors per bit to prevent the information from being disturbed when read.
Capacity increases from
static, dynamic to magnetic disk
virtual memory involves the operating
system unlike cache memory
VA(VPN, OFfset) find number of bits offset?
take size of pages 2^12 page 12 is nuber of bits in offset
Looking at a lot more misses column wise(100%)
than row wise (25%) So fixed is in the middle. Rows : a lot less misses with columns a lot of misses Two best cases:scanning rows (minimal miss rate) of two matric es
If the second-level cache contains the desired data, the miss penalty for the fi rst-level cache will be essentially
the access time of the second-level cache, which will be much less than the access time of main memory
The use of a larger block decreases the miss rate and improves the efficiency of
the cache by reducing the amount of tag storage relative to the amount of data storage in the cache.
Cache index:
the index field of any address of a cache block must be that block number
SO LOWER 3 BITS ARE THE BLOCK ADDRESS/INDEX AND UPPER 2 BITS ARE TAGS. If two blocks maps to the same location, .
the last one survives.
Where to look in the cache for each possible address:
the low-order bits of an address can be used to find the unique cache entry to which the address could map
The total size of the cache in blocks is equal to
the number of sets times the associativity.
The index value is used to select the set containing the address of interest, and
the tags of all the blocks in the set must be searched.
The tag needs only to contain
the upper portion of the address, corresponding to the bits that are not used as an index into the cache. we need only have the upper 2 of the 5 address bits in the tag, since the lower 3-bit index field of the address selects the block.
Because DRAMs use only a single transistor per bit of storage,
they are much denser and cheaper per bit than SRAM.
Virtual Memory:
to allow efficient and safe sharing of memory among multiple programs, such as for the memory needed by multiple virtual machines for cloud computing, and to remove the programming burdens of a small, limited amount of main memory
SRAM needs only minimal power
to retain the charge in standby mode. As long as we have power, we have the value
Flash Memory
type of electrically erasable programmable read-only memory (EEPROM)/tiny little storage cards writes can wear out flash memory bits.
This second-level cache is normally on the same chip and is accessed
whenever a miss occurs in the primary cache.
For a fixed cache size, increasing the associativity decreases the number of sets
while increasing the number of elements per set.
Disks have a slower access time because they are mechanical devices—flash is 1000 times as fast and DRAM is 100,000 times as fast—.
yet they are cheaper per bit because they have very high storage capacity at a modest cost—disk is 10 to 100 time cheaper
When CPU performance increased - Miss penalty becomes more significant (why?)
• Decreasing base CPI - Greater proportion of time spent on memory stalls • Increasing clock rate - Memory stalls account for more CPU cycles
Choosing Which Block to Replace
• Direct mapped: no choice • Set associative - Prefer non-valid entry, if there is one - Otherwise, choose among entries in the set • Least-recently used (LRU) - usage based - Choose the one unused for the longest time • Simple for 2-way, manageable for 4-way, too hard beyond that
The GOAL: illusion of large, fast, cheap memory • How do we create a memory that is large, cheap and fast (most of the time)? 1. Hierarchy 2. Parallelism
• Fact: Large memories are slow (dram), fast memories are small (sram) • Fact: Large memories are cheap (disk), fast memories are expensive (dram)
