Coverage is My Friend

 

The Stimulus Package

When my friend first muttered these words, I thought he decided to leave the rewarding world of verification and become a political pundit. I was dismayed because I don’t like the term pundit, it rhymes with bandit. Ever wondered why we have verification gurus and they have political pundits? In any case, it turned out it was about a far less controversial kind of stimulus package, the one you feed to your design to tickle its inputs and make its transistors go crazy. Phew!

However, once my mind wandered towards the halls of power, I discovered, strangely enough, that there are a lot of similarities between the government stimulus and the one that’s the bread and butter of verification engineers. To begin with, both of them are a big deal. If you live in the US and turn on the radio, chances are somebody is talking about how massive the economic stimulus is. Well, if you want to verify anything these days, you’re probably going to throw massive stimuli at it. To manage their stimulus package, the government has set up a large bureaucracy; you’re going to set up a large testbench for yours. Your testbench will take a lot of time and effort to develop and debug and once you’re done with it, you’ll want to reuse it over and over again. Similarly, it takes a while to get large bureaucracies off the ground but once they’re in business they tend to find a way to reuse themselves over and over again. Next, there’s the question of when you should stop stimulating and that’s when things get really interesting. See, the government hasn’t yet figured out how to design uncontroversial cover targets and measure how well their stimulus hits them and that’s why the benefits of the stimulus package continue to be widely debated. We did, my friend can vouch for that.

I could keep going but this is not a town hall meeting. Let me just say that in verification, we like our stimulus to have both an evenly distributed effect on the design to make sure we sensitize as many paths as possible, as well as a more focused effect on the areas that implement the main functionality of the DUT. As a result, there is a place for both randomly generated stimuli as well as more targeted ones. Although it may be a stretch to call the economic stimulus random (no, let’s not go there), its stated intention is to spread it in a reasonable manner to stimulate as many economic paths as possible. There is also the equivalent of a directed stimulus and is commonly referred to as pork-barrel but the main topic of this blog is to talk about the kind of stimulus that you want to spread around. And that is where stimulus coverage enters the scene. Often maligned, derided or simply ignored, stimulus coverage is the ugly duckling of the family, far less glamorous than its brothers who capture complex arbitration scenarios or exotic packet combinations. Today, my friend asked me to make a case for bringing stimulus coverage out of the shadows and into the limelight of our coverage strategy.

I will start by first pointing out that we need to make sure that our stimulus is legal (I’m not going to go into what that means for the government one), so we spend a great deal of time developing constraints to ensure that. These constraints become very complex very quickly partly because the environments in which the verified blocks operate are getting more and more complex and partly because we strive to capture only realistic scenarios on the inputs of our blocks because we don’t want to send the block into the twilight zone and we don’t want to waste valuable simulation cycles. So now that we’ve spent all this time developing these complex constraints, how do we know that the outcome is the kind of stimulus we were shooting for? Try functional cover groups that capture what we think is reasonable stimulus, aka stimulus coverage. You may object that this is double work because it boils down to expressing the same stimulus intent once implicitly through the constraint set and once explicitly through the stimulus coverage. I think it is well worth it. After all, the whole existential purpose of the testbench is to sensitize potential bugs and to make sure you observe them. Stimulus is the one that takes care of the first part so whatever we can do to make it more robust will pay back dividends many times over. I wish the same will be the case with the economic stimulus…
The second thing to keep in mind is that the constraint solvers are not perfect. There is no guarantee that they will always provide a solution if one exists and moreover, the solution may not be exactly what you expect. When a range of solutions can be generated, how do you guarantee that the solver distributes the values the way you imagined it or that it touches the “interesting” values in that range? Again, stimulus coverage can help. Once you explicitly documented what the key values or ranges are through the stimulus coverage targets, you need to make sure you cover them all. For that, you can manually play around with distributions, add constraints, etc. Not rocket science by any means but potentially very messy. Very well suited for an automated tool as my friends keeps pointing out 😉

To conclude, stimulus is a crucial part of verification (and the economic recovery some say) and any effort we spend in baking it better will be well worth it. Stimulus coverage is a straightforward way to address that. Obviously it will not guarantee that you will sensitize every corner case bug in the design but it will at least make sure you have a reasonable starting point. Go back to the government’s stimulus. Ideally, it would help every business in the country in some way. Yes, we can? Neh, sorry Mr. President that’s too much to ask but one step in the right direction is for instance to send some money to every state. If you don’t send any money to say California, it’s close to certain you’re not going to help too many businesses there so the first thing to do is to make sure every state is covered. Kind of like our stimulus coverage. Clearly, there is much our governments can learn from verification engineers so maybe one day our gurus will become their pundits.

I hope I didn’t let my friend down and was persuasive enough to send you writing stimulus coverage right away and if the Echo of my blog will ever so feebly resonate in your verification approach, my friendship will be stronger than ever.

  • del.icio.us
  • Digg
  • Facebook
  • Google Bookmarks
  • Print
  • Twitter
  • StumbleUpon