Comments on: Let 100 flowers bloom ….. roses or weeds?

By: Adam Donlin

Adam Donlin — Fri, 16 May 2008 05:22:11 +0000

As mentioned in the previous comments, it’s difficult to have a scientific discussion around these issues when there are so few data to examine.

Who holds the burden of evidence here? Is it OSCI? Have they done enough to convince the modeling-public that SystemC models have sufficient horsepower to compete with other approaches? Is it possible to answer these questions in a public forum without breaching a clause in a software license?

Are we using the correct metrics to compare these different approaches? Is it all just about simulation speed or should we be looking at the product of simulation speed /and/ development effort? Are these alternative modeling styles even universally applicable or are we just talking about better ways to hack an ISS?

If I may abuse Grant’s metaphor a little: lets not forget that doors have locks. Is that the true value of “the other side of the door”? Who is locked in? Who is locked out?

By: Grant Martin

Grant Martin — Thu, 08 May 2008 22:53:33 +0000

I got a comment from my friend and colleague Frank Schirrmeister who is over at Synopsys (I first worked with Frank many years ago in the Alta group of Cadence. He is Director, Product Marketing, System-Level Solutions for Synopsys). He wanted to clarify Synopsys’s position regarding OSCI’s TLM 2 standard:

“Synopsys is fully committed to TLM-2 adoption, period! After contributing key virtual platform technology components to the OSCI SystemC TLM 2.0 standardization efforts in January 2007 our team very actively contributed to the specification process. As outlined in slide 2 of our NASCUG 2008 presentation our TLM 2.0 strategy for the system-level solutions is based on interoperability to be able to offer (1) standard-compliant, simulator-agnostic TLM IP in our DesignWare® System-Level Library, (2) authoring tools for model and platform creation with our Innovator tool suite and (3) comprehensive design services for model and virtual platform creation. We will roll out product support – both for tools and libraries – later this year.”

By: Jakob Engblom

Jakob Engblom — Mon, 05 May 2008 19:48:50 +0000

I think benchmarks here are really tricky to do well… what do you want to actually compare? Overall efficiency of a particular simulator when models are optimized for it? Or the efficiency for the case of simulating SystemC in particular? At which levels of detail (or abstraction)? What to measure? Target instructions per second, target transactions per second, target time progress per second? It is a very slippery thing, in my experience.

I have done a lot of benchmarking of our simulation solution, and even within the scope of a single kernel, single system, single level of abstraction doing anything meaningful is awfully hard. Measurements indicating that one simulation design is better than another are often invalidated by changing the nature of how the target software is using a system. As a simple example, in the same system, performance can vary by a factor of a hundred or more between a benchmark doing intense device accesses and one doing processor-intense integer math.

Maybe the best way would be meta-benchmarks, with designs to model in the best way possible for a particular simulator. With a full target software stack to drive it… But even so, finding a common base platform to work from is pretty hard. What should it be? Something old and establihsed like a PowerPC 750 with a cpc700 chipset running Linux 2.4? Or a naked plain MIPS32 processor with memory, no chipset, and just a device?

More questions than answers, sorry about that.

By: Grant Martin

Grant Martin — Mon, 05 May 2008 01:54:24 +0000

Steve, that’s an interesting suggestion, although another alternative, and one that might be easier to start up, would be for OSCI to take a lead here in at least encouraging some standard test cases that could be used to benchmark simulation approaches. On the other hand, that might be regarded as beyond its remit in that it concentrates on the language itself and use models. Let’s see what others think (if there are more comments).

By: Steve Leibson

Steve Leibson — Mon, 05 May 2008 01:44:59 +0000

Grant, it would appear to me that there’s a need for an EEMBC-like organization to benchmark and compare the efficiency of different simulation interfaces on different products. Perhaps Markus Levy could be enticed to collaborate.

By: Jakob Engblom

Jakob Engblom — Sat, 03 May 2008 19:34:21 +0000

I agree with Brian that the other interfaces are just as important as the memory-mapped interface. But I do not agree that Imperas are the only ones to take it on…

Pretty much anyone who has ever created a TLM model has wound up modeling some non-memory-mapped interfaces in some way. Sometimes falling back to SystemC channels (basically wires in a different shape), but also often doing function-call-driven truly transaction-level modeling. But due to the need to abstract form the physical layer, each such model has been done in a different way. Hampering interoperability of models.

In any case, the focus needs to be on how to interconnect models in a range of languages and from different sources, rather than on SystemC per se. It is a modeling language among many, good for some things, less good for others. And not to forget that models quite often need to come in binary form in order to protect someones idea of intellectual property…

Another body looking into this is actually Power.org, with its Technical SubCommittee (TSC) on Virtual Platforms and Simulation. See http://www.power.org/resources/committeeview/

By: Brian Bailey

Brian Bailey — Fri, 02 May 2008 20:33:30 +0000

Grant has done a great job at identifying the various strategies in use for creating simulation models and showing the diversity amongst them. I would counter by saying it is surprising that there are not more solutions and approaches to the problem and this is probably only because people do not have the imagination to come up with new things - instead trying to copy the successes of the past. But lets get back to why I think there are so many. This is simply because we have not yet decided what we want to do with them. Ask users why they use a virtual platform and you will get a plethora of answers ranging from functional verification, to validations of architectural decisions, to performance evaluation, to software verification and so on. Is it reasonably to expect a single solution to be the best at all of these. No. They have different needs, different priorities and different users in many cases. Grant I believe does hit on the biggest problem and that is the one of communications between models. At RT level we solved this with simple semantics for interfaces attached to some low level communications channels - called wires. The semantics dealt with value, transition etc. We have failed to define the equivalent for higher levels of abstraction and even things like TLM are only attempting to do it for a small class of interfaces - those of memory mapped devices. But what about all of the other interfaces in a system? As far as I know, Imperas is the only one to have attempted to take that on. Without these interfaces, we cannot expect systems models to come together, and thus I would say that until we decide on interface standards, the whole virtual platform market will not grow significantly. How many kernels should there be - does anyone care? This is an artifact of the implementation and in itself provides no functionality.