For those interested, I highly recommend looking at the slides and recorded presentations form the Neuromorphic-Inspired Computational Elements Conference - http://nice.sandia.gov/videos.html

It started last year and was expanded greatly for this past February's conference to a 3-day event.

There were many outstanding talks there on many levels and describing many variations of the approaches all aimed at developing 'brain-inspired' alternatives to our Von Neumann/Turing architecture in place now for almost 70 years.

In my mind, the best individual approaches they and several others have described over the past few years cover most of the topics brought up at RC2 but in a much more comprehensive and cohesive manner. That is, these approaches contain the key elements of 'approximate computing', 'augmented CMOS', and of course neuromorphic.

There are other even more exciting and capable research projects going on which weren't discussed at this event, but I collectively there has been tremendous progress over just the past couple of years which has generated a strong new momentum which in turn will further accelerate the work to develop the next computing platform. Conferences, journals, workshops, and new funded programs have been developing to fuel this momentum. Despite all of this, most people it seems aren't yet fully aware of all of this activity, but that issue too is starting to be addressed as the main stream media has begun to pick up on this emerging world.

It's an exciting time to be involved in, and although similar in feeling to the way we felt in the 70's and 80's when both computing and communications went 'personal', the possibilities being opened by these new approaches are many times greater.

Background reading of value

"For those attending the IEEE Rebooting Computing Workshop this week with an interest in neuromorphic computing, we recommend:

 

'Finding a roadmap to achieve large neuromorphic hardware systems' by Jennifer Hansler and Bo Marr (http://journal.frontiersin.org/Journal/10.3389/fnins.2013.00118/full).  This paper provides interesting and relevant material for the workshop that is expected to help guide our discussion."

 

Dave Mountain, Facilitator for the Neuromorphic Computing discussion at RC2

Approximate Computing: A Path beyond the Tyranny of Digital Computing

by Hadi Esmaeilzadeh

Over the course of past four decades, consistent and exponential improvement in transistor scaling coupled with continuous advances in general-purpose processor design has exponentially reduced the cost of raw material for computing, i.e., performance; making it a pervasive commodity. While in 1971, at the dawn of microprocessors, 1 MIPS (Million Instruction Per Second) cost $5000, today the same level of performance costs less than 5¢. This exponential reduction in the cost has made the computing industry, the industry of new capabilities. We are not an industry of replacement whose economic cycle relies on consumers replacing inventory when they run out of products. It is even hard to perceive running out of a software app such as Microsoft office. Instead, the computing industry’s economic ecosystem relies on continuously providing new capabilities both at the device and at the service level.

Before the effective end of Dennard scaling, we consistently improved performance and efficiency while maintaining generality in general-purpose computing. As the benefits from scaling diminish and the current paradigm ofthe microprocessor design, multicore processors, significantly falls short ofthe traditional cadence of performance, we are facing an “iron triangle”; we can only choose any two of performance, efficiency, and generality at the expense of the third. Energy efficiency now fundamentally limits microprocessor performance gains. These shortcomings may drastically curtail computing industry from continuously delivering new capabilities, the backbone of its economic ecosystem. Hence, developing solutions that improve performance and efficiency, while retaining as much generality as possible, are of outmost importance.

Radical departures from conventional approaches are necessary to provide energy-efficacy and large performance gains for a wide range of applications and domains. One such departure is general-purpose approximate computing, where error in computation is acceptable and the traditional robust digital abstraction of near-perfect accuracy is relaxed. Conventional techniques in energy-efficient computing navigate a design space defined by the two dimensions of performance and energy, and traditionally trade one for the other. General-purpose approximate computing explores a third dimension, error, and trades the accuracy of computation for gains in both energy and performance.

This radical approach in general-purpose computing will only be beneficial if a large body of applications can tolerate error during execution. Fortunately, as the landscape of computing is changing toward providing more personalize and more targeted experience for the users, a vast number of emerging applications are inherently approximate and error-resilient. These applications can be categorized to four classes:

(1) Applications with analog inputs (wearable electronics, voice recognition, scene reconstructions).

(2) Applications with analog output (multimedia).

(3) Applications with multiple possible answers (machine learning, web search, heuristics).

(4) Convergent applications (big data analytics, optimizations).

More importantly, in this realm of computing, the rate of data generation and collection is growing overwhelmingly beyond what conventional computing platforms can process. By trading off computation accuracy for gains in performance and efficiency, general-purpose approximate computing aims to exploit this emerging opportunity in the application level to tackle the aforementioned fundamental challenges in the transistor and architecture level. One may visualize these trade-offs as finding the Pareto-optimal points in the processor design space, as shown below. Traditionally, for any set of workloads, the set of possible processor implementations may be plotted, with energy efficiency on one axis and performance on the other, and the best implementations residing on the two-dimensional frontier. When approximation is supported, the degree of permissible error represents a third axis. The Pareto surface in this three-dimensional space represents the best points of performance, efficiency, and error. However, this surface is not yet well understood. Navigating this three dimensional space provides many opportunities for innovation across the entire system stack.

As an instance, analog circuits inherently trade accuracy for significant gains in energy-efficiency. However, it is challenging to utilize them in a way that is both programmable and generally useful. In our most recent work that will be presented in International Symposium on Computer Architecture (ISCA) on June 2014, we propose a solution—from circuit to compiler—that enables general-purpose use of limited-precision, analog hardware to accelerate “approximable” code—code that can tolerate imprecise execution. We utilize an algorithmic transformation that automatically converts approximable regions of code from a von Neumann model to an “analog” neural model.   The core idea is to learn how aregion of approximable code behaves and automatically replace the original code with an efficient computation of the learned model. The neural transformation of general-purpose approximable code provides an avenue for realizing the benefits of analog computation while targeting code written in conventional languages. The insights from this work show that it is crucial to expose analog circuit characteristics to the compilation and neural network training phases. At run time, while the processor executes the program, it invokes a reconfigurable accelerator, which we named Neural Processing Unit (NPU), instead of running the original region of code. Our most recent work reports on the design and integration of a mixed-signal NPU for general-purpose code execution. The NPU model offers a way to exploit analog efficiencies, despite their challenges, for a wider range of applications than is typically possible. Further, mixed-signal execution delivers much larger savings for NPUs than digital. Analog neural acceleration provides whole application speedup of 3.7× and energy savings of 6.3× with quality loss less around 10%. Even though the results are very encouraging, there are still several challenges that need to be overcome. The full range of applications that can exploit mixed-signal NPUs is still unknown, as is whether it will be sufficiently large to drive adoption in high-volume microprocessors. It is still an open question how developers might reason about the acceptable level of error when an application undergoes an approximate execution including analog acceleration. Finally, in a noisy, high-performance microprocessor environment, it is unclear that an analog NPU would not be adversely affected. However, the significant gains from A-NPU acceleration and the diversity of the studied applications suggest a potentially promising path forward. This work also shows how relaxing the abstraction of near-perfect accuracy can provides a bridge between two disjoint models of computing, neuromorphic and von Neumann.

Despite its great potential, practical and prevalent use of general-purpose approximate computing requires techniques that seamlessly integrate with the current well-established practices of programming and system design and provide a smooth and evolutionary adaptation path for this revolutionary paradigm in computing.

In general, when conventional approaches run out of steam, it is time for extreme creativity. In fact, we may be living the most exciting era of computing!

Hadi Esmaeilzadeh is the Catherine M. and James E. Allchin Early Career Professor of Computer Science at Georgia Institute of Technology. His dissertation received the 2013 William Chan Memorial Dissertation Award from University of Washington. He founded the Alternative Computing Technologies (ACT) Lab, where he works with his students on developing new technologies and cross-stack solutions to develop the next generation computing systems for emerging applications. Hadi received his Ph.D. in Computer Science and Engineering from University of Washington in 2013. He has a Master’s degree in Computer Science from The University of Texas at Austin (2010), and a Master’s degree in Electrical and Computer Engineering from University of Tehran (2005). Hadi received the Google Research Faculty Award in 2013.

Hadi’s research is recognized by three Communications of the ACM Research Highlights and three IEEE Micro Top Picks. His work on dark silicon has been profiled in New York Times.

Approximate computing

In an effort to set up a community around approximate computing, we are asking participants in Rebooting Computing that are knowledgeable about this topic to post references to explanatory material.

Approximate computing is an approach to  low-energy computing. There are general approaches toward lowering the energy consumption of a computer that would have the side effect of reducing the reliability of the computation and results. This effect is mitigated in approximate computing through algorithms that tolerate uncertainty where it is unimportant to the answer and bolster uncertain information through codes and other algorithmic methods.

We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.

Neuromorphic processing

In an effort to set up a community around adiabatic and reversible computing, we are asking participants in Rebooting Computing that are knowledgeable about this topic to post references to explanatory material.

The scope of this thread can include both analog and digital brain-inspired or neuromorphic approaches.

Living brains use chemical and electrical analog behavior in synapses and soma (nerve cell bodies) to compute in the ways needed for survival.The analog scope of this thread is artificial systems with a human-engineered device or circuit taking the place of a synapse neuron. These systems are sometimes called "neuromorphic."

However, Artificial Neural Networks (ANNs) can be implemented digitally using fixed or floating point values in lieu of analog signals. The digital scope of this thread would be devices and architectures that would execute the neural algorithms with speed and energy efficiency much higher than possible with a von Neumann computer.

We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.

Extending CMOS technology direction

In an effort to set up a community around extending CMOS, we are asking participants in Rebooting Computing that are knowledgeable about this topic to post references to explanatory material.

We are defining this topic area to include topics at or beyond the current frontier of technology. There appears to be a consensus that CMOS and Moore's Law will continue only for a finite period of time, with this blog thread focused on what comes beyond that time. To be true to the phrase "rebooting computing," the systems envisioned by this thread should include a new active device that has circuit behavior distinctly different from a transistor, a new circuit distinctly different from CMOS, or a new device that serves a special role in a non-von Neumann architecture.



We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.

update of low-power image recognition competition

Committee Members:

These are the current members in the committee.  If you want to join (or you want to recommend someone), please send email to me (yunglu@purdue.edu). Thank you.

• Alex Berg (UNC)
• David Kirk (Nvidia)
• Yung-Hsiang Lu (Purdue)
• Gi-Joon Nam (IBM, representative of ACM SIGDA)

We are inclined to remove the idea of using a display for giving the test data.  This will introduce too much uncertainty in data acquisition through cameras. Instead, we plan to give the data to participants through networks directly.

The competition will be held in mid 2015.  The rules will be announced in summer 2014.