Chapter 5: Source Internals

Simulation Kernel Deep Dive

How evaluate-update-delta scheduling makes concurrent C++ models behave like hardware.

18 minLesson 2

How to Read This Lesson

This is a source-reading lesson. We will use the Accellera implementation as a microscope, while keeping the LRM as the portability contract.

The SystemC scheduler is a discrete-event simulation kernel. Its job is to decide which process runs, when updates become visible, which events wake new work, and when simulation time can advance. This is strictly defined by the IEEE 1666 Language Reference Manual (LRM), primarily in Section 4 on Elaboration and Simulation Semantics.

Source and LRM Trail

This lesson is deliberately source-facing. Use Docs/LRMs/SystemC_LRM_1666-2023.pdf to decide what must be portable, then use .codex-src/systemc/src/sysc and .codex-src/systemc/src/tlm_core to see one reference implementation. Treat private members as explanatory, not as APIs your models should depend on.

The Shape of a Simulation Step

According to the LRM, the simulation semantics are broken down into distinct phases: Evaluation, Update, Delta Notification, and Time Advance. We can observe these phases in action by writing a complete model that tracks delta cycles.

#include <systemc>
#include <iostream>
 
using namespace sc_core;
 
SC_MODULE(DeltaTracker) {
  sc_signal<bool> sig_a{"sig_a"};
  sc_signal<bool> sig_b{"sig_b"};
 
  SC_CTOR(DeltaTracker) {
    SC_THREAD(stimulus);
    SC_METHOD(monitor_a);
    sensitive << sig_a;
    dont_initialize();
 
    SC_METHOD(monitor_b);
    sensitive << sig_b;
    dont_initialize();
  }
 
  void stimulus() {
    std::cout << "[Time: " << sc_time_stamp() << ", Delta: " << sc_delta_count() << "] Stimulus: writing sig_a=1\n";
    sig_a.write(true);
    
    wait(SC_ZERO_TIME); // Advance one delta cycle
    
    std::cout << "[Time: " << sc_time_stamp() << ", Delta: " << sc_delta_count() << "] Stimulus: writing sig_b=1\n";
    sig_b.write(true);
    
    wait(10, SC_NS); // Advance time
    
    std::cout << "[Time: " << sc_time_stamp() << ", Delta: " << sc_delta_count() << "] Stimulus: done\n";
  }
 
  void monitor_a() {
    std::cout << "[Time: " << sc_time_stamp() << ", Delta: " << sc_delta_count() << "] monitor_a triggered: sig_a=" << sig_a.read() << "\n";
  }
 
  void monitor_b() {
    std::cout << "[Time: " << sc_time_stamp() << ", Delta: " << sc_delta_count() << "] monitor_b triggered: sig_b=" << sig_b.read() << "\n";
  }
};
 
int sc_main(int argc, char* argv[]) {
  DeltaTracker tracker("tracker");
  sc_start();
  return 0;
}

Evaluate Phase

During the evaluation phase, the kernel selects a ready process and resumes its execution. A ready SC_METHOD runs to completion. A ready SC_THREAD (or SC_CTHREAD) resumes until it calls wait() or returns. The LRM explicitly states that the order of process execution during the evaluation phase is non-deterministic. Your models must not rely on the order in which processes execute within the same delta cycle.

Processes can write to signals, notify events, call TLM transport functions, and update ordinary C++ state. But primitive channel updates (like writes to sc_signal) are deferred.

Update Phase

After all ready processes have been evaluated (the set of runnable processes is empty), the kernel enters the update phase. Every primitive channel that had request_update() called during the evaluate phase now has its update() function executed.

For sc_signal, this is when the new value actually becomes the current value. By deferring the update, SystemC guarantees that all processes reading the signal during the evaluate phase read the old value, regardless of process execution order. This avoids race conditions and mirrors the behavior of hardware registers.

Delta Notification Phase

During the update phase, or directly from processes using zero-delay notification (notify(SC_ZERO_TIME)), events may be scheduled for the next delta cycle.

If there are pending delta notifications, the kernel advances to the next delta cycle (incrementing sc_delta_count()) while keeping the simulation time constant, and moves those notified processes back to the runnable set. It then loops back to the Evaluate phase.

This gives SystemC a way to settle combinational behavior without advancing time.

Time Advance Phase

When the runnable set is empty and there are no more primitive channel updates or delta notifications, the current time step is fully settled. The scheduler then looks for the next earliest timed event (e.g., a process waiting for 10 ns or a delayed notification). Simulation time jumps directly to that time, the relevant events are triggered, processes become ready, and the loop starts again at the Evaluate phase.

Why wait() Needs Process State

SC_THREAD can suspend and resume. That means the implementation needs a process control mechanism (such as coroutines, fibers, or user-level threads like QuickThreads in the reference implementation) that preserves the call stack and execution state across waits. wait() returns control to the kernel, and the kernel context-switches back when the thread resumes.

Debugging With the Kernel Model

When a model behaves strangely, classify the problem based on the LRM phases:

  • Evaluation issue: wrong sensitivity, method ran too early, method initialized unexpectedly.
  • Update issue: signal write not visible until update phase. Did you read immediately after writing in the same process?
  • Delta issue: event notified in a later delta than expected, causing off-by-one delta delays.
  • Time issue: timed wait or clock period mismatch.
  • Termination issue: no timed events remain (simulation ends prematurely), or a process never yields (infinite loop blocking the evaluate phase).

Under the Hood: The sc_simcontext::crunch() Loop

The SystemC scheduler operates inside sc_simcontext::crunch(). It is a strict state machine:

  1. Evaluate Phase: The kernel pops processes from the m_runnable list and executes them. For an SC_METHOD, it calls the function. For an SC_THREAD, it performs a context switch via QuickThreads to resume the thread.
  2. Update Phase: The kernel iterates over m_update_list (channels that had request_update() called) and invokes update().
  3. Delta Notification Phase: Events notified with zero delay (SC_ZERO_TIME) are processed. Sensitive processes are added to the m_runnable list. If the list is non-empty, the delta cycle repeats.
  4. Time Advance Phase: If m_runnable is empty, the kernel advances simulation time (m_curr_time) to the timestamp of the next earliest event in m_timed_events, and triggers those events.

IEEE 1666-2023 LRM: Formal Elaboration and Simulation Semantics

To truly master the SystemC simulation kernel, we must look at the formal rules established by the IEEE 1666-2023 Language Reference Manual (LRM). Chapter 4 of the LRM strictly defines Elaboration and Simulation Semantics. Let's map these formal rules to the practical behavior we observed above and dive into the specific clauses that govern the Accellera source code.

Elaboration Phase (LRM Section 4.1)

Before simulation starts (via sc_start), the application undergoes Elaboration. Elaboration is the construction of the module hierarchy, instantiation of ports and channels, and the registration of processes. The LRM defines this phase in multiple distinct steps:

  1. Instantiation: Modules, ports, primitives, and hierarchical channels are instantiated. The constructors of all SC_MODULE elements execute.
  2. Process Registration: Calls to SC_METHOD, SC_THREAD, and SC_CTHREAD macros register the member functions with the simulation kernel (sc_simcontext). Static sensitivity is established via the sensitive stream.
  3. Port Binding: Ports are bound to channels or other ports (LRM 4.1.3). The kernel checks port binding rules recursively until every port is bound to a channel or interface.
  4. End of Elaboration Callbacks: The end_of_elaboration() callback is invoked on all instantiated modules and channels (LRM 4.1.4). This allows objects to execute initialization logic that depends on the fully constructed hierarchy but must run before simulation begins.

Any attempt to dynamically spawn processes (via sc_spawn) during elaboration is governed by strict rules, but generally, the hierarchy must be static by the time the kernel transitions to simulation.

The Simulation Kernel Phases (LRM Section 4.2.1)

The LRM defines the simulation loop as a sequence of deterministic steps operating on non-deterministic process execution. The state machine defined in the standard maps directly to the Accellera SystemC reference implementation.

Initialization Phase (LRM 4.2.1.1)

When sc_start() is called, the kernel executes the Initialization Phase precisely once:

  1. Initialize Values: The initial values of all primitive channels are evaluated.
  2. Start of Simulation Callbacks: start_of_simulation() is invoked on all modules and channels.
  3. Process Execution: Every registered method and thread process is made runnable (added to the initial evaluate queue) unless dont_initialize() was called on the process during elaboration.
  4. Update Phase Execution: Any updates requested during initialization (e.g., initial signal writes) are executed.
  5. Delta Notifications: Any zero-delay notifications are triggered.

Evaluate Phase (LRM 4.2.1.2)

The Evaluate Phase is where the "heavy lifting" happens. The kernel maintains a set of runnable processes.

  • Rule of Non-determinism: The LRM states that the order of execution of ready processes in the evaluate phase is undefined. The Accellera implementation often pops processes from a LIFO stack or a FIFO queue depending on the internal kernel version, but you must never rely on this order.
  • Execution Semantics:
    • A runnable SC_METHOD executes from the beginning of its registered function until it returns.
    • A runnable SC_THREAD (or SC_CTHREAD) resumes from its last suspension point (wait()) and executes until it suspends again or terminates.
  • Immediate Notifications: If a process executes notify() (with no time argument) on an sc_event, it triggers an immediate notification (LRM 4.2.1.2). Any process sensitive to this event is immediately added to the set of runnable processes in the current evaluate phase. This can potentially cause an infinite loop if processes immediately trigger each other without yielding.

Update Phase (LRM 4.2.1.3)

After the set of runnable processes becomes empty, the Evaluate phase terminates. The kernel transitions to the Update Phase.

  • Primitive Channels: During the evaluate phase, processes may have called request_update() on primitive channels. The kernel iterates through these channels exactly once and calls their virtual update() method.
  • State Resolution: For sc_signal, the update() method checks if the newly proposed value differs from the current value. If it does, the current value is overwritten, and a value-changed event is notified.

Delta Notification Phase (LRM 4.2.1.4)

After the Update Phase finishes, the kernel checks for delta notifications.

  • Zero-delay Notifications: If an event was notified using notify(SC_ZERO_TIME) or if a signal value change triggered an event, these are evaluated.
  • Process Wakeup: Processes sensitive to these events are marked as runnable.
  • Delta Cycle Loop: If the set of runnable processes is non-empty after evaluating delta notifications, the kernel increments the delta cycle count (sc_delta_count()) and immediately loops back to the Evaluate Phase. Time does not advance.

Time Advance Phase (LRM 4.2.1.5)

If the delta notification phase yields an empty set of runnable processes, the kernel checks for pending timed notifications.

  • Finding the Next Event: The kernel inspects the queue of timed events (e.g., notify(10, SC_NS) or wait(10, SC_NS)) and determines the earliest future timestamp.
  • Advancing Time: Simulation time (sc_time_stamp()) is advanced to this timestamp.
  • Waking Timed Processes: All events scheduled for this new time are triggered, moving sensitive processes back to the runnable set.
  • Loop: The kernel increments the delta cycle counter and returns to the Evaluate Phase.
  • Termination: If there are no pending timed events, or if the simulation time reaches the maximum time passed to sc_start(), simulation finishes. end_of_simulation() callbacks are invoked.

Deep Dive: Accellera SystemC Source Implementation (sysc/kernel)

Understanding the LRM is crucial, but looking at how the Accellera source code implements these rules grounds our knowledge in reality. The simulation loop lives within the sysc/kernel directory.

The Brain of the Simulator: sc_simcontext

The central class coordinating all simulation is sc_simcontext (found in sysc/kernel/sc_simcontext.cpp). When you call sc_start(), you are ultimately invoking sc_get_curr_simcontext()->simulate(), which delegates to the crunch() function.

Let's peek at the conceptual structure of the crunch() loop inside the Accellera implementation:

// Conceptual representation of sc_simcontext::crunch() in sysc/kernel/sc_simcontext.cpp
inline void sc_simcontext::crunch( bool once ) {
    while ( true ) {
        // 1. EVALUATE PHASE
        while ( m_runnable->is_empty() == false ) {
            sc_process_b* next_process = m_runnable->pop_front();
            
            // Execute the process (calls run() on method or thread)
            m_current_writer = next_process;
            next_process->semantics();
            m_current_writer = 0;
        }
 
        // 2. UPDATE PHASE
        if ( !m_update_list->is_empty() ) {
            sc_update_mark_t* update_ptr = m_update_list;
            m_update_list = 0; // Clear for next cycle
            // Iterate and call update() on primitive channels
            while( update_ptr ) {
                update_ptr->update();
                update_ptr = update_ptr->next;
            }
        }
 
        // 3. DELTA NOTIFICATION PHASE
        if ( m_delta_events->is_empty() == false ) {
            // Process SC_ZERO_TIME notifications
            m_delta_events->process_events();
        }
 
        // 4. CHECK FOR NEXT CYCLE OR TIME ADVANCE
        if ( m_runnable->is_empty() ) {
            // No delta events fired, break loop to advance time
            break; 
        }
 
        // Increment delta count before looping back to Evaluate
        m_delta_count++;
    }
}

Process Management: sc_process_b, sc_method_process, sc_thread_process

The Accellera implementation models processes using a base class sc_process_b (defined in sysc/kernel/sc_process.h).

  • sc_method_process: This class represents an SC_METHOD. Its semantics() execution simply invokes the user's C++ function. Because it cannot yield, it requires no execution stack state management.
  • sc_thread_process: This class represents an SC_THREAD. Managing its execution requires an underlying coroutine library. Historically, Accellera SystemC uses QuickThreads (found in sysc/qt). When sc_thread_process::semantics() is called, the kernel initiates a thread context switch (saving the kernel's CPU registers and stack pointer, and restoring the thread's). When the thread calls wait(), it saves its state and context switches back to the kernel.

The Update Mechanism: sc_prim_channel

How does the kernel know which signals to update? All primitive channels inherit from sc_prim_channel (sysc/communication/sc_prim_channel.h). When a process writes to an sc_signal, the signal's write() method calls request_update().

The request_update() function adds the channel's pointer to the kernel's m_update_list queue. This list is drained and cleared at the start of the Update Phase, ensuring every channel is updated exactly once per delta cycle, no matter how many times it was written to during the Evaluate Phase.

Event Notification: sc_event

The sc_event class (sysc/kernel/sc_event.cpp) is the backbone of synchronization. When you call e.notify(), the implementation immediately iterates through the event's lists of statically and dynamically sensitive processes and pushes them onto the m_runnable queue.

When you call e.notify(SC_ZERO_TIME), the event is pushed onto the kernel's m_delta_events list, deferring the wakeup until the Delta Notification Phase.

When you call e.notify(10, SC_NS), the event is wrapped in a structure and inserted into a priority queue (m_timed_events) ordered by future execution time.

Why this Architecture Matters for Modeling

Understanding this architecture is not just academic; it dictates how you write robust hardware models:

  1. Lock-Ups and Infinite Loops: If you write an SC_METHOD that contains a while(true) loop without a wait() (which is illegal in methods anyway), you trap the simulation kernel inside the Evaluate Phase forever. Time will never advance.
  2. Immediate vs. Zero-Delay Notifications: Using immediate notify() creates an immediate feedback loop. If Process A immediately notifies Event X, and Process B is sensitive to X, B runs in the same evaluate phase. If B then modifies state that triggers Process A, you have a zero-delay infinite loop that never even reaches the Update Phase. Always prefer delta notifications (notify(SC_ZERO_TIME)) or channel updates (sc_signal) for feedback loops to allow the kernel to breathe and resolve state.
  3. Wait State Overhead: Because SC_THREADs require memory allocations for thread stacks and context switching overhead (QuickThreads), excessive use of threads can slow down simulations. This is why high-performance TLM models predominantly use b_transport (which borrows the caller's thread) and SC_METHODs for synchronization.

By understanding the IEEE 1666 rules and the Accellera implementation, you can debug lockups, race conditions, and performance bottlenecks by mentally tracing the sc_simcontext::crunch() loop.

Deep Dive: Accellera Source for sc_signal and update()

The sc_signal<T> channel perfectly illustrates the Evaluate-Update paradigm of SystemC. In the Accellera source (src/sysc/communication/sc_signal.cpp), sc_signal inherits from sc_prim_channel.

The write() Implementation

When you call write(const T&), the signal does not immediately change its value. Instead, it stores the requested value in m_new_val and registers itself with the kernel:

template<class T>
inline void sc_signal<T>::write(const T& value_) {
    if( !(m_new_val == value_) ) {
        m_new_val = value_;
        this->request_update(); // Inherited from sc_prim_channel
    }
}

The request_update() call appends the channel to sc_simcontext::m_update_list.

The update() Phase

After the Evaluate phase finishes (all ready processes have run), the kernel iterates over m_update_list and calls the update() virtual function on each primitive channel. For sc_signal, this looks like:

template<class T>
inline void sc_signal<T>::update() {
    if( !(m_new_val == m_cur_val) ) {
        m_cur_val = m_new_val;
        m_value_changed_event.notify(SC_ZERO_TIME); // Notify processes sensitive to value_changed_event()
    }
}

This guarantees that all concurrent processes see the same old value until the delta cycle advances, perfectly mimicking hardware register delays.

Comments and Corrections