What is a P-state?

When someone refers to a P-state, generally only the frequency is talked about. For example, on my Intel Core Duo, P0 is 2.3 GHz, and P1 is 980 MHz. In truth, a P-state is both a frequency and voltage operating point. Both are scaled as the P-state increases. 

The effect of reducing frequency

 It's obvious that performance is directly related to frequency. We all know that increasing the frequency, increases a processor's performance. The same applies for decreasing the frequency. If you halve the frequency, a compute bound task runs half as fast. For example, if your task is compute bound, requiring 100 % of the CPU for 1 second at 2 GHz, it will take 2 seconds to execute at 1 GHz. (This is roughly correct. There are a host of other factors influencing runtime, such as cache size and speed, interrupts, etc.) 

Now wait a moment. If we reduce the frequency, increasing the runtime of an application, how does this increase battery life? If we are decreasing the frequency, we are increasing the CPU utilization and reducing the % idle time. See the frequency half of Figure A. This shouldn't have any effect on the power usage of the processor. It's running all that time anyway. 

You're right, historically. This was the case years ago with those ancient generations old processors. Though I don't know this for a fact, I suspect that in some cases, decreasing the percent idle time might have increased energy usage since increased processor activity increases instantaneous power, and we're decreasing the % idle time (i.e. time of low activity). This is where voltage scaling comes into play. There are two primary reasons for P-states, one is to reduce the peak thermal load, and the other is to save power. 

Reducing peak thermal load

 The reasons why you want to reduce the peak thermal load is pretty obvious. The instantaneous energy usage (power) of the processor is related to its activity. If the processor is very busy, requiring a lot of gates to do a lot of switching, it runs hotter. So reducing the frequency reduces the peak thermal output even if the total energy usage is not reduced. The advantage of reducing peak thermal load has to do with cost. The effectiveness of your cooling is based upon peak power, not average power. (I'm neglecting the effect of thermal inertia.) So if you can reduce peak power, you reduce the cost and size of the equipment having to do the cooling.  

There's got to be a catch

There are 5 CC-states and, depending upon how you count, 6 PC-states in the Penryn line of Intel processors. And, in Microsoft XP, there are 4 OS C-states. So are there 5 C-states, 6 C-states, 4 C-states, or 15 C-states? Choose the number that you are least uncomfortable with. Personally, I first imagine a 3 set Venn diagram with overlapping elements and added transition annotations. Then I get confused and give up. 

Given what we've talked about above, it seems as if we should always drop a core into the lowest permissible CC-state, right? 

There are a few reasons for not doing this. First off, the OS's Power Management (PM) policy, and not the hardware, determines when a core enters a CC-state. From our standpoint as a hardware manufacturer, we have little to do with this. I'll talk about why this is important later. Secondly, there is always a cost for dropping into a lower C-state. That cost is the amount of time required for the core to transition from an idle state, e.g. CC5, to C0. As you start using deeper CC-states, latency becomes significant. For example, the latency to go from CC3 to CC0 is around 20 us, literally ages when we're talking about 3 GHz processors. 

This latency penalty is even worse once you realize that the phrase, "in a given C-state," is misleading. As I've mentioned above, it's easy to think of a core as descending as a waterfall from C0 into C1 into C2 into C3. (See Figure A.) If this were the case, you'd suffer only one 20 usec penalty. No, it's oscillating between C0 and C3 hundreds, if not thousands, of times a second until the OS's PM code decides that the percentage residency merits ascending / descending to the next C-state (e.g. CC3 to CC2). In Windows, the C-state that a core transitions to is based on the % idle over a certain interval. This means that each transition exacts that 20 usec penalty, and there are hundreds of transitions. Doing the math, experiencing a 20 usec delay 100 times a second is a whopping 2 msec of added latency per second. (See Figure B.) 

Even if it is possible to drop a core into a deeper CC-state, the OS has to ask itself various questions, such as what is the likelihood that processes are going to be doing more work very soon, so that dropping into a deeper CC-state might actually cost an unacceptable penalty? Similarly, the processor has to ask whether dropping a core into a lower CC-state is going to cause incorrect operation, say whether the delay in the processing of an interrupt will cause an event to be lost.

Figure A. The waterfall misconception.

Figure B. What actually happens "in a given C-state".

So what's this thing about "C-states, C-states and even more C-states"? It turns out that there are different kinds of C-states depending upon what part of your system you are talking about. There are core C-states, processor C-states, and OS C-states. All are similar and are idle states (I'm excluding C0, of course.) They are also different in some substantial ways.

A core C-state is a hardware C-state. There are several core idle states, e.g. CC1 and CC3. As we know, a modern state of the art processor has multiple cores, such as the recently released Core Duo T5000/T7000 mobile processors, known as Penryn in some circles. What we used to think of as a CPU / processor, actually has multiple general purpose CPUs in side of it. The Intel Core Duo has 2 cores in the processor chip. The Intel Core-2 Quad has 4 such cores per processor chip. Each of these cores has its own idle state. This makes sense as one core might be idle while another is hard at work on a thread. So a core C-state is the idle state of one of those cores.

A processor C-state is related to a core C-state. At some point, cores share resources, e.g. the L2 cache or the clock generators. When one idle core, say core 0, is ready to enter CC3 but the other, say core 1, is still in C0, we don't what the fact that core 0 is ready to descend into CC3 to prevent core 1 from executing because we just happened to shut down the clock generators. Thus we have the processor / package C-state, or PC-state. The processor can only enter a PC-state, say PC3, if both cores are ready to enter that CC-state, e.g both cores are ready to step into CC3. I'll talk more about this in a subsequent section.

A logical C-state: The last C-state is the OS's view of the processors' C-states. In Windows, a processor's C-state is pretty much equivalent to a core C-state. In fact, the OS's lower level power management software determines when and if a given core enters a given CC-state using the MWAIT instruction. There is one important difference. When an application, such as Intel's PowerInformer, thinks it's interrogating a processor core CC-state, what is returned is the C-state of what is called a "logical core". (A logical core is technically not the same as a physical core. In my experience, a logical core is almost always the same as a physical core, but it doesn't have to be.) Logical cores don't have to worry about little things such as the hardware the OS is running on. For example, the C-state of a logical core doesn't worry about the barriers imposed by shared resources, such as the clock generators, I talked about earlier. Logical Core 0 can be in C3 while Logical Core 1 is in C0.

This seems a little confusing doesn't it? So how do logical core C-states, core C-states and processor C-states relate to each other? Take the situation above: From the OS perspective, logical core 0 is in C3 and logical core 1 is in C0. Since C3, from the hardware perspective, actually shuts down a shared process, the clock generators, (physical) core 0 must be held at CC2 since core 1 is in C0 and using the clock generators. The processor, in a global sense, is not idle since core 1 is in C0, so the processor's C-state is C0. To use a little bit of that intimidating mathematics,

Processor C-state = Min(core C-states)

Core C-state = Minimum barrier(set of all logical C-states)

Logical C-state = anything the OS wants

Next: There has got to be a catch

With the exception of C0, where the CPU is active and busy doing something, a C-state is an idle state. Since an idle CPU isn't doing anything (i.e. any useful work), why not shut it down? No one is going to notice since there's no one using it. (Letting a Penryn run at full bore when idle is like driving in circles very fast; all you're doing is going nowhere quickly.)

A P-state is an operational state, meaning that the core / processor can be doing useful work in any P-state. The most obvious example is when your laptop is using a low power profile and operating on battery. The OS will lower the C0 operating frequency and voltage, i.e. enter a higher P-state. Reducing the operating frequency reduces the speed at which the processor operates, and so the energy usage per second (i.e. power). Reducing the voltage decreases the leakage current from the CPU's transistors, making the processor more energy efficient resulting in further gains. The net result is a significant reduction in the energy usage per second of the processor. On the flip side, an application will take longer to run. This may or may not be a problem from a power perspective. I'll talk about this issue in some depth in a later blog.

C-states and P-states are also orthogonal. This is a fancy mathematical term meaning that each can vary independently of the other. This doesn't mean that in the higher C-states, the voltage doesn't change. It only means that when you resume C0, you go back to the operating frequency and voltage defined by that P-state.

Next time: C-states, C-states and even more C-states

So if you would be so kind as to "constructively" comment. Feel free to point out bad wording, areas that need clarification, even places where you thinking that I'm just plain wrong. Of course, if you think I'm wrong, just don't say, "Hey man, your wrong." Be constructive. Say something like, "Hey man, you're wrong. But check out the cool Heckledorf article on Wintel PM." And if you want to go so far as to write a correction, who am I to complain?

I appreciate all the feedback you can give me. I may crawl off and sulk a little bit, but I'll be back to make the corrections.

Sincerely,

Taylor 

=============

INTRODUCTION: THE POWER MANAGEMENT ELEMENTS There are many ways you can divide up responsibilities for power management (PM). I use the following division:

1. Processor
2. Chipset
3. Platform
4. Operating System (not user settable policy)
5. Operating System (user settable policy), and
6. Application

At least initially, this paper will address Processor PM (#1), and a little bit of not-user-settable Operating System PM (#4).

Processor Power Management is that which is within the processor, namely within that large black chip with that handsome Intel logo that presides over the motherboard. This type of PM includes the powering down of different parts of the processor itself, e.g. things such as the cores and cache.

Chipset Power Management is that which extends one step beyond the processor itself, to the immediately surrounding circuitry that works intimately with the processor. This set of peripheral chips is typically made by the same manufacturer, e.g. Intel.

Platform Power Management involves the remaining parts of the computer, such as the other components on the motherboard outside of the chipset and processor, and the peripherals hung off of the system such as wireless interfaces and hard drives.

Operating System (OS) Power Management:

That the user can influence: These are what MS Windows refers to as the power profiles, e.g. Super Power Saver mode. Whereas lower-level OS power management deals directly with the processor and chipset, this higher-level mostly involves peripherals, such as how long the LCD remains active and the HD spun up. This influence amounts to suggestions to the OS Power Management software.

That the user can't influence: In the new Intel systems, the interaction between the OS and the processor's Power Management can be very intimate. For example, the interaction between Microsoft Vista/XP is very closely coupled with core C-state, the OS directly making some of the most significant power management decisions relative to the processor. Though software controlled, Vista/XP does not allow the user to directly influence this type of policy.

Application Power Management, the last in the list, isn't going to be addressed in this paper outside of this brief description. This type of PM addresses how applications can change their behavior to minimize power consumption. To say it another way, these are the actions applications can take to facilitate and not defeat the other types of power management. Though it won't be discussed here, an application can be written that will unnecessarily keep the processor in the highest power state. As it turns out, this author is mostly involved in such power management.

This paper discusses mostly processor and lower-level OS power management, i.e. #1 and #4 above. These are less understood but, arguably, the most important to application writers. Equipped with such knowledge, SW engineers can insure that their products do not defeat these most effective types of PM. As it turns out, many applications and OSs are programmed in such a way that these most important energy saving features are partially defeated. I'll discuss this further in a latter section.

This paper does not discuss what many would claim is one of the most important PM domains, namely that of embedded processors. Embedded processors, e.g. the more modern ARM processors, have sophisticated PM capabilities, though they take a different approach than do Intel processors. This discussion, though very interesting and important, is not within the scope of this paper. 

UP NEXT WEEK: P-States and C-states, same menagerie, different animals

'nough said.

Jason: "what can software developers do while they creating the applications to use those hardware components to consume the power? (sic) "

There are a variety of things. Intel's focus is on processors, of course. Intel processors, such as the Core Duo, have different operating states (P-state) and power conservation states (C-states). In some future blog, I'll them in painful detail.

Having P-states are a way for the processor to save energy. Technically, it is a frequency and voltage operating point: the lower the frequency, the lower the performance and the better the energy savings. Up to a point, the same applies to voltage. Note that no matter the P-state, the processor is fully operational. Here’s an example from one of my portables. When on AC-power, it operates at a frequency of 2333 MHz. (I don’t know the voltage.) On battery, it operates at 979.9 MHz. In both states, the processor is fully functional, though the lower frequency P-state uses less power. From my standpoint as the user, my computer games run slower and my compiles take longer.

In contrast, a C-state is (approximately) a per core power conservation state. If idle, a core can drop into different C-states, i.e., it can shut down parts of itself. The higher (i.e. deeper) the C-state, the more is shut down, and the greater the power savings. This is possible because the core is idle in all but C0. For example, let’s look at the Core Duo: in C0, the core is fully operational; in C1, it is in a “HALT” state; in C2, L1 caches are flushed; etc.

(Though it isn’t technically correct, here’s how I remember which state is what: P-state for processor state, and C-state for core state. No doubt, the “P” and “C” refer to something completely different, but this is how I remember which means what.)

Let’s get back to Jason’s original question: what can an app developer do? There are several things, mostly involving letting the processor (P-state) and cores (C-state) enter deeper power saving states.

(1) Get the job done faster. This little observation surprised me in the beginning. I’d assumed that if I paced my app, like I’d do if I were in a race, I would use less power. This little bit of anthropomorphizing led me down the wrong path. (By the way, .I also name my cores. Core 0 is Timmy and the other is Fred.) What you optimize is the amount of time the cores and processor spend in energy conserving states. If you pace your application, stretching it out over the entire interval of execution (say, by using a lower P-state), you can conceivably never let the cores shut down into the deeper C-states. If you get that dang job done faster, a core might be breathing harder, but it’s going to have a chance to drop into the deeper C-states. This is one of those wonderful situations when the there’s no cost / benefit tradeoff to worry about. (This mostly applies for time bound apps, and other apps, such as a streaming app, with lower duty cycles.

(2) Use interrupts instead of polling. Busy waiting keeps the cores active, preventing them from entering deeper C-states.

(3) Use the longest timer interrupt interval you can get away with. Many app developers figure that the shorter the timer interval, the better the performance. This is true only up to a point. If it takes your app 250 msec to a job done, setting the timer to 10 m sec isn’t going to buy you any better performance. It’s just going to keep the cores busy, not letting them drop into deeper C-states.

(4) Disable or destroy timers after use. Invoking the timer handler when it is no longer needed can keep the processor running when it doesn’t have to, preventing the cores from dropping into deeper C-states.

(5) Turn off external devices when no longer needed. Again, invoking unnecessary interrupt handling can keep the cores from dropping into a lower C-state.

(6) Balance your threads. This is an efficiency problem. Ideally in most cases balancing your threads will lead to better performance when you have multiple cores, since P states are dependent, if one core is used 100% while other is only 10%, you are wasting compute power on the other core, since P-state for both is going to be the highest. (I can actually think of some pathological cases when this might not be the case, but that’s a topic for another day.) For time bound apps, since the job can be finishes faster with balanced threading, it also allows a deeper P and C-state than otherwise. Let me explain. Below a certain C-state, shared resources are being shut down. So if we have an unbalanced situation where core 0 could possibly descend into a C3 state, but core 1 cannot, core 0 can’t enter C3. This is because C3 requires the shutting down of a resource shared by both cores.

(7) Buffer I/O. Allow your CD, DVD or HD to spin down.

(8) Minimize data movement. This allows localization of data, permitting more efficient caching. If data is localized and in L2, then memory accesses from off processor devices don’t have to wake up the cores, letting the cores remain in a deeper C-state.

(9) Make your app energy aware. Here’s an example. MS XP and Vista have various counters that give you battery, C-state and P-state information. Conceivably, you can write your app such that it operates one way if the notebook is tethered to that wall outlet, and another, if on battery. You might also vary your app’s behavior if you find that you have less than 30 minutes of battery power left.

I'm starting up a new blog concerning power issues and application software.

I'm Taylor Kidd, an AE (that's Applications Engineer in Intelispeak) in the Intel SSG (Software Solutions Group), more specifically, the PPE (Power Platform Enabling) team.

Our mission, since we decided to accept it, is to make venders and those others with an interest, aware of power issues having to do with their client platforms. (A client platform is what you and I think of as a computer. Servers, also computers, are typically heard (e.g. on webpages) but not seen, being hidden away behind bland doors with card triggered locks, and non-descript warehouse-looking buildings located in urban industrial jungles.)

I have to tell you that my group only deals with the client, and more specifically, with the PC side of the power problem. This is one facet of the power equation. Two other very important pieces of the power puzzle are the server side (can anyone say Google?), and the small footprint embedded side (e.g. cell phones).

The power problems on the Server side are well known. There is also a lot of focused economic incentive to solve or mitigate these problems. Server farms are both big in size and big in business. They have very little directly to do with you and me. (Personally, I can't think of even one person I know who has a 1000 machine server farm in their basement.) Server farms guzzle energy as a result of operating the servers themselves, and performing the cooling of those servers. This translates to a lot of out of pocket cash. And it's noticeable. Let's take a look at Google. A Wikipedia article ("Google Platform", http://en.wikipedia.org/wiki/Google_platform) estimates that Google maintains close to half a million servers spread throughout the world. If we figure an energy cost of $50 per server per month (in total electrical burden) per server ("Energy Hogs on the Server Farm", David Raths, http://www.govtech.com/gt/102970?id=&story_pg=2), this amounts to a cost of around $25M per month, or $300M a year. I have got to tell you that if I was looking at an electricity bill of $300M per year, I'd have a might powerful reason to do something about it.

It may be a surprise to a lot of you that in absolute terms, increasing the power efficiency of PC clients is going to make a whooping bigger difference in terms of green house gases, amount of oil and coal burnt, and waste heat produced, then all the server farms in the world. It is estimated that the number of PCs in the world exceeds one billion. ("PC numbers set to hit 1 billion", by Siobhan Chapman, http://www.techworld.com/news/index.cfm?NewsID=9119) A government estimate figures that between $25 and $75 a year could be saved per PC by simply activating its power management features. (See http://www.energystar.gov/index.cfm?c=power_mgt.pr_pm_step1). So if we can save $50 per PC per year -- using the completely unscientific guess that if I split the difference and lump notebooks and desktops together, I get $50 per PC -- this amounts to a savings of $50 billion dollars per year. And this is just by employing existing power management features that already exist. What would the savings be if we saved even more power through using energy efficient applications and other techniques?

I know exactly what you're thinking, because I thought the same thing. "Holy moly, Batman! It sound like it makes a lot of sense to make PCs more energy efficient. What are we waiting for? It's energy clobbering time!" (Yes, I know that I'm mixing up my superhero quotes.)

Wait a minute, youngster. Let's look at it from another perspective. A $50 savings per year is going to result in a savings of a whooping $4 per month (assuming 1 PC per household). I've got to tell you that I'm not going to notice any $4 savings on my bill, especially when it's "potential savings" that I can't really internalize by way of my pocket book.

My point is that the magnitude of the problem posed by server farms is dwarfed by that produced by the energy waste of desktops and notebooks.

So my conclusion? It makes sense from a societal point of view to make PCs operate more efficiently. But the average Joe isn't going to bat an eye at the fact that he "might" save an ethereal $4 a month.

So where exactly this line of thought going? Are we ever going to get to application software and its potential for saving power? Stay tuned for the next exciting adventures of Application Power Man!

Disclaimer: Though this blogger did not invent his figures, and indeed obtained them form various web sources, he can in no way guarantee that those web sources did not themselves invent the figures.