Q-RASP: A Novel Framework to Improve Information Flow Across Network-on-Chip Architectures

At a Glance

Researchers at Colorado State University have developed an improved framework for routing information while avoiding network congestion on network-on-chip architectures, called Q-RASP. This technique was compared against other routing policies in synthetic and real application traffic. It can reduce packet latency by up to 18.3% and has up to 6.7% lower energy consumption than other Q-routing policies.

Background

Network-on-chip architectures are the leading solution to manage data movement bottlenecks in manycore chips. Their routing policies determine the path a packet takes towards a destination. Simple policies, such as those that always choose the same route for packets between a source and destination may not perform as well as adaptive policies that can better distribute movement and traffic to avoid data congestion. Many adaptive policies lack information about congestion across the whole network. Q-routing is an attractive method for creating a global estimation of congestion and learning a routing policy that minimizes packet latency. However, previous approaches to applying Q-routing have underperformed due to variables being slow to calculate or failures to accurately represent the congestion.

Overview

A new Q-learning based network-on-chip routing framework has been developed called Q-Routing for NoCs with Region-Awareness and Shared Path Experience (Q-RASP). Previous limitations of Q-routing have been related to the cost function and mechanism to update information. Q-RASP introduces a regional congestion cost that pushes the policy to find paths through less congested regions and allows quick shifts to a nearby optimal route when a part of a path becomes congested. To consistently find optimal routes, updates must be made frequently. To increase updates and gain a better understanding of total network congestion, a Shared Path Experience mechanism provides information about information packets traveling to different destinations, but that use the same route. When compared against other routing policies in synthetic traffic, including other Q-routing policies, Q-RASP showed lower or comparable average packet latency. Q-RASP also showed a reduction in the total energy consumption per packet compared with the energy consumption of other Q-routing policies.

Benefits

Congestion-aware optimization reduces hardware overhead compared to latency-aware optimization
Additional updates allow faster convergence of Q-values
Lower or comparable packet latency compared to other routing policies for bit-reversal, butterfly, transpose, and random traffic patterns – improved by up to 18.3%
Lower total energy consumption per packet than other Q-routing policies – improved by up to 6.7%

Applications

Network-on-chip architecture information routing policies

Publications

Kamil Khan and Sudeep Pasricha. 2023. A reinforcement learning framework with region-awareness and shared path experience for efficient routing in networks-on-chip. IEEE Design & Test. doi: 10.1109/MDAT.2023.3306719

Last Updated: February 2024

Opportunity

Available for Exclusive Licensing
TRL: 3

Inventors

Sudeep Pasricha
Kamil Khan

Reference Number

2024-017

Licensing Manager

Jessy McGowan
Jessy.McGowan@colostate.edu
970-491-7100