Operating System-Aware Cache Optimization Techniques for Multi Core Processors

Hasina Khatoon Prof. Dr. Shahid Hafeez Mirza Prof. Dr. Talat Altaf
FIT 2011 CISD NEDUET 1

Overview
Background Multi Core Processor Issues Possible Solutions Role of Operating System Implications on Cache Optimization Techniques Review References
FIT 2011 CISD NEDUET 2

Overview
Background Multi Core Processor Issues Possible Solutions Role of Operating System Implications on Cache Optimization Techniques Review References
FIT 2011 CISD NEDUET 3

Background
Implications of Moores Law Increasing number of cores on a single chip
 Parallelism at the chip level

Problems and Issues emerged Memory Wall has grown higher

 Requires a Paradigm shift
FIT 2011 CISD NEDUET 4

Issues in Multi Core Processors

Processor-Memory Speed Gap - Memory Wall Software and Algorithms Mechanisms for Chip Management Power Management Heterogeneous or Homogenous Cores On-chip Interconnection Network Managing Design Complexity Others
FIT 2011 CISD NEDUET 5

Overview
Background Multi Core Processor Issues Possible Solutions Role of Operating System Implications on Cache Optimization Techniques Review References
FIT 2011 CISD NEDUET 6

Possible Solutions
Solution lies at all levels of execution environment
 Hazelwood and Zahran    
FIT 2011

(SIGOPS 2009) [1]

Layers of Execution Environment

On-chip memory hierarchy The operating system Compiler Programming environment
CISD NEDUET 7

Layers of execution environment

FIT 2011

CISD NEDUET

Solution to the Memory Wall Couple the two important layers of execution environment
 Cache Optimization Techniques  Operating System Policies and Mechanisms

Make Operating System Aware of Cache Optimization Techniques Make operating aware of multiple cores
FIT 2011 CISD NEDUET 9

Overview
Background Multi Core Processor Issues Possible Solutions Role of Operating System Implications on Cache Optimization Techniques Review References
FIT 2011 CISD NEDUET 10

Role of Operating System Multi core-aware operating systems built from scratch
 Factored Operating System (fos) [2]  Barrelfish Operating System [10, 30]

Operating system components

 Schedulers that are core-aware
G-LOMARC-TS [4]

On-chip Resource Management

 QoS-Aware Resource Management
FIT 2011 CISD NEDUET

[12]
11

Operating Systems for CMPs

FIT 2011

CISD NEDUET

12

Operating Systems for CMPs

FIT 2011

CISD NEDUET

13

Overview
Background Multi Core Processor Issues Possible Solutions Role of Operating System Implications on Cache Optimization Techniques Review References
FIT 2011 CISD NEDUET 14

Operating System and Cache Optimizations

PMUs provide feedback to OS
 OS performs optimum cache partitioning among cores [14]

L2-Aware Scheduling Algorithms

 Group and Gang Scheduling
[16]

Shared cache space allocation based on sharing behaviour of workloads

 Use RapidMRC as the tool
FIT 2011 CISD NEDUET

[23]
15

Cache-Aware OS Improvements

FIT 2011

CISD NEDUET

16

OS-Aware Cache Optimizations Prefetching in L2 with hints from operating system

 Avoid useless prefetching

Victim cache to save evicted blocks

 Improve performance of NUCA cache

Avoid conflict misses using dynamic page mapping policies

FIT 2011 CISD NEDUET 17

OS-Aware Cache Oprimizations

FIT 2011

CISD NEDUET

18

Overview
Background Multi Core Processor Issues Possible Solutions Role of Operating System Implications on Cache Optimization Techniques Review References
FIT 2011 CISD NEDUET 19

Review
Background Issues in Multi Core Processors Solution to the Memory Wall Operating System Policies and Mechanisms Implications on Cache Optimization Techniques Couple the two
FIT 2011 CISD NEDUET 20

Overview
Background Multi Core Processor Issues Possible Solutions Role of Operating System Implications on Cache Optimization Techniques Review References
FIT 2011 CISD NEDUET 21

References
[1] K. Hazelwood, M. Zahran, Challenges and Opportunities at All Levels -Interactions Among Operating Systems, Compilers and Multi core Processors, ACM SIGOPS Operating System Review, Volume 43, Issue 2, April 2009, pp 3-4 [2] D. Wentzlaff, A. Agarwal, The Case for a Factored Operating System, SIGOPS Operating System Review, 43(2), 2009, pp 76-85 [3] D. Nellans, R. Balasubramonian, E. Brunvard, A Case for Icreased Operating System Support for Chip Multiprocessors, Proceedings of 2nd IBM Watson P=ac 2, 2005 [4] X. Zeng, A. Sodan, Job Scheduling with Lookahead Group Matchmaking for Time/Space Sharing on Multicore Parallel Machines, Proceedings of 14th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), 2009, pp 232-258
FIT 2011 CISD NEDUET 22

References
[5] B. Ren, System Design for Chip Multiprocessors, A PhD Thesis Proposal, University of Cambridge, July 2004 [6] A. Baumann et al., The Multikernel: A new OS architecture for scalable multicore systems, Proceedings of the ACM SIGOPS 22nd Symposium on Operating System Principles, 2009, pp 29-44 [7] S. B. Wickizer et al., Corey: An Operating System for Many Cores, Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation, 2008, pp43-57 [8] O. Mutlu, T. Moscibroda, Parallelism-Aware Batch Scheduling: Enabling High-performance and Fair Shared Memory Controllers, IEEE Micro, January/February 2009, pp22-32 [9] Z. Fang, X. H. Sun, Y. Chen, Surendra Byna, Core-Aware Memory Access Scheduling Schemes, IEEE International Parallel and Distributed Processing Symposium, 2009, pp 1-12
FIT 2011 CISD NEDUET 23

References
[10] A. Baumann et al., Your Computer is Already a Distributed System. Why isnt your OS? Proceedings of the 12th Conference on Hot Topics in Operating Systems, 2009 [11] T. Shimosawa, Y. Ishikawa, Inter-kernel Communication between Multiple Kernels on Multi core Machines, IPSJ Online Transactions, Volume 2, Dec 2009, pp261-279 [12] H. Kannan et al., From Chaos to QoS: Case Studies in CMP Resource Management, ACM SIGARCH Computer Architecture News, Volume 35, Issue 1, March 2007, pp 21-30 [13] T. Nojiri, Y. Kondo, N. Irie, M. Ito, H. Sasaki, H. Maejima, Domain Partitioning Technology for Embedded Multicore Processors, IEEE Micro, November/December 2009, pp7-17

FIT 2011

CISD NEDUET

24

References
[14] R. Azimi, D. K.Tam, L. Soares, M. Stumm, Enhancing Operating System Support for Multi core Processors by Using Hardware Performance Monitoring Units, ACM SIGOPS Operating System Review, Volume 43, Issue 2, April 2009, pp 56-65 [15] S. Co, L. Jin, Managing Distributed, Shared L2 Caches through OS-Level Page Allocation, Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, 2006, pp 455-468 [16] A. Fedorova, M. Seltzer, C. Small, D. Nussbaum, Performance of Multithreaded Chip Multiprocessors and Implications for the Operating System Design, Proceedings of USENIX Annual Technical Conference, 2005 [17] P. Denning, The Working Set Model for Program Behavior, Communications of the ACM, Volume 11, Issue 55, May 1968, pp 323-333.
FIT 2011 CISD NEDUET 25

References
[18] G. Suo, X. Yang, Balancing Parallel Applications on Multi-core Processors Based on Cache Partitioning, IEEE International Symposium on parallel and Distributed Processing with Applications (ISPA), 2009, pp 190-195 [19] J. Lin, Q. Lu, X. Ding, Z. Zhang, Xiaodong Zhang, P. Sadayappan, Gaining Insights into Multi core Cache Partitioning: Bridging the Gap between Simulation and Real Systems, IEEE 14th International Symposium on High Performance Computer Architecture (HPCA), 2008, pp367-378 [20] A. Fedorova, S. Blagodurov, S. Zhuravlev, Managing Contention for Shared Resources on Multi core Processors, ACM Queue, Volume 8, Issue 1, January 2010, pp 11-30 [21] D. Chandra, F. Guo, S. Kim, Y. Solihin, Predicting InterThread Cache Contention on a Chip Multi-Processor Architecture, Proceedings of the 11th International Symposium on High-Performance Computer Architecture (HPCA-11), 2005, pp 340-351
FIT 2011 CISD NEDUET 26

References
[22] B. Zhou, J. Qiao, S. Lin, Research on Dynamic Cache Distribution Scheduling Algorithms on Multicore Processors, International Conference on E-Business and Information System Security, 2009, pp 1-4 [23] D. Tam, Operating System Management of Shared Caches on Multicore Processors, a PhD Thesis, Graduate Department of Electrical and Computer Engineering, University of Toronto, 2010 [24] X. Zhang, Operating System-Level On-Chip Resource Management in The Multicore Era, a PhD Thesis, Department of Computer Science, University of Rochester, New York, 2010 [25] E. Ebrahimi, O. Mutlu, C. J. Lee, Y. N. Patt, Coordinated Control of Multiple Prefetchers in Multi-Core Systems, MICRO 2009 [26] N. Chen, R. Johnson, Patterns for Cache Optimizations on Multi-Processor Machines, Workshop at the 2nd Annual Conference on Parallel Programming Patterns (ParaPlop), 2010
FIT 2011 CISD NEDUET 27

References
[27] J. Hennessy, D. Patterson, Computer Architecture A Quantitative Approach, Fourth Edition, Morgan Kaufmann Publishers, 2006, p. 293-310; C22-C38 [28] T. Romer, D. Lee, B. Bershad, B. Chen, Dynamic Page Mapping Policies for Cache Conflict Resolution on Standard Hardware, Proceedings of the First USENIX Symposium on Operating Systems Design and Implementation (OSDI), 1994, pp 255-266 [29] M. Jahre, L. Natvig, Performance Effects of a Cache Miss handling Architecture in a Multi-core Processor, Norwegian Informatics Conference (NIK), 2007 [30] A. Schpbach et al., Embracing diversity in the Barrelfish many core Operating System, Proceedings of the Workshop on Managed Many-Core Systems (MMCS), 2008

FIT 2011

CISD NEDUET

28

Questions?

FIT 2011

CISD NEDUET

29

Thank You!

FIT 2011

CISD NEDUET

30