For those interested, I highly recommend looking at the slides and recorded presentations form the Neuromorphic-Inspired Computational Elements Conference - http://nice.sandia.gov/videos.html
It started last year and was expanded greatly for this past February's conference to a 3-day event.
There were many outstanding talks there on many levels and describing many variations of the approaches all aimed at developing 'brain-inspired' alternatives to our Von Neumann/Turing architecture in place now for almost 70 years.
In my mind, the best individual approaches they and several others have described over the past few years cover most of the topics brought up at RC2 but in a much more comprehensive and cohesive manner. That is, these approaches contain the key elements of 'approximate computing', 'augmented CMOS', and of course neuromorphic.
There are other even more exciting and capable research projects going on which weren't discussed at this event, but I collectively there has been tremendous progress over just the past couple of years which has generated a strong new momentum which in turn will further accelerate the work to develop the next computing platform. Conferences, journals, workshops, and new funded programs have been developing to fuel this momentum. Despite all of this, most people it seems aren't yet fully aware of all of this activity, but that issue too is starting to be addressed as the main stream media has begun to pick up on this emerging world.
It's an exciting time to be involved in, and although similar in feeling to the way we felt in the 70's and 80's when both computing and communications went 'personal', the possibilities being opened by these new approaches are many times greater.
IEEE Rebooting Computing is a working group developing new ideas to design future computers. The purpose of this blogger is to exchange ideas and to determine the actions toward meeting the goals of building future computers. If you want to be a contributor, please contact Yung-Hsiang Lu (yunglu@purdue.edu).
Wednesday, June 25, 2014
Monday, May 12, 2014
Background reading of value
"For those attending the IEEE Rebooting Computing
Workshop this week with an interest in neuromorphic computing, we recommend:
'Finding a roadmap to achieve large neuromorphic hardware systems' by Jennifer
Hansler and Bo Marr (http://journal.frontiersin.org/Journal/10.3389/fnins.2013.00118/full).
This paper provides interesting and relevant material for the workshop that is
expected to help guide our discussion."
Dave Mountain, Facilitator for the Neuromorphic Computing discussion at RC2
Tuesday, April 29, 2014
Approximate Computing: A Path beyond the Tyranny of Digital Computing
Over the course of past four decades, consistent and
exponential improvement in transistor scaling coupled with continuous advances
in general-purpose processor design has exponentially reduced the cost of raw
material for computing, i.e., performance; making it a pervasive commodity. While
in 1971, at the dawn of microprocessors, 1 MIPS (Million Instruction Per
Second) cost $5000, today the same level of performance costs less than 5¢. This
exponential reduction in the cost has made the computing industry, the industry
of new capabilities. We are not an industry of replacement whose economic cycle
relies on consumers replacing inventory when they run out of products. It is
even hard to perceive running out of a software app such as Microsoft office. Instead,
the computing industry’s economic ecosystem relies on continuously providing
new capabilities both at the device and at the service level.
Before the effective end of Dennard scaling, we consistently
improved performance and efficiency while maintaining generality in general-purpose
computing. As the benefits from scaling diminish and the current paradigm ofthe microprocessor design, multicore processors, significantly falls short ofthe traditional cadence of performance, we are facing an “iron triangle”; we
can only choose any two of performance, efficiency, and generality at the
expense of the third. Energy efficiency now fundamentally limits microprocessor
performance gains. These shortcomings may drastically curtail computing
industry from continuously delivering new capabilities, the backbone of its
economic ecosystem. Hence, developing solutions that improve performance and
efficiency, while retaining as much generality as possible, are of outmost
importance.
Radical departures from conventional approaches are
necessary to provide energy-efficacy and large performance gains for a wide
range of applications and domains. One such departure is general-purpose approximate
computing, where error in computation is acceptable and the traditional robust
digital abstraction of near-perfect accuracy is relaxed. Conventional techniques
in energy-efficient computing navigate a design space defined by the two dimensions
of performance and energy, and traditionally trade one for the other.
General-purpose approximate computing explores a third dimension, error, and
trades the accuracy of computation for gains in both energy and performance.
This radical approach in general-purpose computing will only
be beneficial if a large body of applications can tolerate error during
execution. Fortunately, as the landscape of computing is changing toward
providing more personalize and more targeted experience for the users, a vast
number of emerging applications are inherently approximate and error-resilient.
These applications can be categorized to four classes:
(1) Applications with analog inputs (wearable electronics,
voice recognition, scene reconstructions).
(2) Applications with analog output (multimedia).
(3) Applications with multiple possible answers (machine
learning, web search, heuristics).
(4) Convergent applications (big data analytics,
optimizations).
More importantly, in this realm of computing, the rate of
data generation and collection is growing overwhelmingly beyond what
conventional computing platforms can process. By trading off computation
accuracy for gains in performance and efficiency, general-purpose approximate
computing aims to exploit this emerging opportunity in the application level to
tackle the aforementioned fundamental challenges in the transistor and
architecture level. One may visualize these trade-offs as finding the
Pareto-optimal points in the processor design space, as shown below.
Traditionally, for any set of workloads, the set of possible processor
implementations may be plotted, with energy efficiency on one axis and
performance on the other, and the best implementations residing on the
two-dimensional frontier. When approximation is supported, the degree of
permissible error represents a third axis. The Pareto surface in this
three-dimensional space represents the best points of performance, efficiency,
and error. However, this surface is not yet well understood. Navigating this
three dimensional space provides many opportunities for innovation across the
entire system stack.
As an instance, analog circuits inherently trade accuracy
for significant gains in energy-efficiency. However, it is challenging to
utilize them in a way that is both programmable and generally useful. In our most recent work that will be presented in International Symposium on Computer
Architecture (ISCA) on June 2014, we propose a solution—from circuit to
compiler—that enables general-purpose use of limited-precision, analog hardware
to accelerate “approximable” code—code that can tolerate imprecise execution.
We utilize an algorithmic transformation that automatically converts
approximable regions of code from a von Neumann model to an “analog” neural
model. The core idea is to learn how aregion of approximable code behaves and automatically replace the original code
with an efficient computation of the learned model. The neural transformation
of general-purpose approximable code provides an avenue for realizing the
benefits of analog computation while targeting code written in conventional
languages. The insights from this work show that it is crucial to expose analog
circuit characteristics to the compilation and neural network training phases.
At run time, while the processor executes the program, it invokes a
reconfigurable accelerator, which we named Neural Processing Unit (NPU),
instead of running the original region of code. Our most recent work reports on
the design and integration of a mixed-signal NPU for general-purpose code
execution. The NPU model offers a way to exploit analog efficiencies, despite
their challenges, for a wider range of applications than is typically possible.
Further, mixed-signal execution delivers much larger savings for NPUs than
digital. Analog neural acceleration provides whole application speedup of 3.7×
and energy savings of 6.3× with quality loss less around 10%. Even though the
results are very encouraging, there are still several challenges that need to
be overcome. The full range of applications that can exploit mixed-signal NPUs
is still unknown, as is whether it will be sufficiently large to drive adoption
in high-volume microprocessors. It is still an open question how developers
might reason about the acceptable level of error when an application undergoes
an approximate execution including analog acceleration. Finally, in a noisy,
high-performance microprocessor environment, it is unclear that an analog NPU
would not be adversely affected. However, the significant gains from A-NPU
acceleration and the diversity of the studied applications suggest a
potentially promising path forward. This work also shows how relaxing the
abstraction of near-perfect accuracy can provides a bridge between two disjoint
models of computing, neuromorphic and von Neumann.
Despite its great potential, practical and prevalent use of
general-purpose approximate computing requires techniques that seamlessly integrate
with the current well-established practices of programming and system design
and provide a smooth and evolutionary adaptation path for this revolutionary
paradigm in computing.
In general, when conventional approaches run out of steam,
it is time for extreme creativity. In fact, we may be living the most exciting
era of computing!
Hadi Esmaeilzadeh is the
Catherine M. and James E. Allchin Early Career Professor of Computer Science at
Georgia Institute of Technology. His dissertation received the 2013 William
Chan Memorial Dissertation Award from University of Washington. He founded the
Alternative Computing Technologies (ACT) Lab, where he works with his students
on developing new technologies and cross-stack solutions to develop the next
generation computing systems for emerging applications. Hadi received his Ph.D.
in Computer Science and Engineering from University of Washington in 2013. He
has a Master’s degree in Computer Science from The University of Texas at
Austin (2010), and a Master’s degree in Electrical and Computer Engineering
from University of Tehran (2005). Hadi received the Google Research Faculty
Award in 2013.
Hadi’s research is recognized
by three Communications of the ACM
Research Highlights and three IEEE
Micro Top Picks. His work on dark silicon has been profiled in New York Times.
Wednesday, March 26, 2014
Approximate computing
In
an effort to set up a community around approximate
computing, we are asking participants in Rebooting Computing that are
knowledgeable about this topic to post references to explanatory
material.
Approximate computing is an approach to low-energy computing. There are general approaches toward lowering the energy consumption of a computer that would have the side effect of reducing the reliability of the computation and results. This effect is mitigated in approximate computing through algorithms that tolerate uncertainty where it is unimportant to the answer and bolster uncertain information through codes and other algorithmic methods.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
Approximate computing is an approach to low-energy computing. There are general approaches toward lowering the energy consumption of a computer that would have the side effect of reducing the reliability of the computation and results. This effect is mitigated in approximate computing through algorithms that tolerate uncertainty where it is unimportant to the answer and bolster uncertain information through codes and other algorithmic methods.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
Neuromorphic processing
In
an effort to set up a community around adiabatic and reversible
computing, we are asking participants in Rebooting Computing that are
knowledgeable about this topic to post references to explanatory
material.
The scope of this thread can include both analog and digital brain-inspired or neuromorphic approaches.
Living brains use chemical and electrical analog behavior in synapses and soma (nerve cell bodies) to compute in the ways needed for survival.The analog scope of this thread is artificial systems with a human-engineered device or circuit taking the place of a synapse neuron. These systems are sometimes called "neuromorphic."
However, Artificial Neural Networks (ANNs) can be implemented digitally using fixed or floating point values in lieu of analog signals. The digital scope of this thread would be devices and architectures that would execute the neural algorithms with speed and energy efficiency much higher than possible with a von Neumann computer.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
The scope of this thread can include both analog and digital brain-inspired or neuromorphic approaches.
Living brains use chemical and electrical analog behavior in synapses and soma (nerve cell bodies) to compute in the ways needed for survival.The analog scope of this thread is artificial systems with a human-engineered device or circuit taking the place of a synapse neuron. These systems are sometimes called "neuromorphic."
However, Artificial Neural Networks (ANNs) can be implemented digitally using fixed or floating point values in lieu of analog signals. The digital scope of this thread would be devices and architectures that would execute the neural algorithms with speed and energy efficiency much higher than possible with a von Neumann computer.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
Extending CMOS technology direction
In
an effort to set up a community around extending CMOS, we are asking participants in Rebooting Computing that are
knowledgeable about this topic to post references to explanatory
material.
We are defining this topic area to include topics at or beyond the current frontier of technology. There appears to be a consensus that CMOS and Moore's Law will continue only for a finite period of time, with this blog thread focused on what comes beyond that time. To be true to the phrase "rebooting computing," the systems envisioned by this thread should include a new active device that has circuit behavior distinctly different from a transistor, a new circuit distinctly different from CMOS, or a new device that serves a special role in a non-von Neumann architecture.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
We are defining this topic area to include topics at or beyond the current frontier of technology. There appears to be a consensus that CMOS and Moore's Law will continue only for a finite period of time, with this blog thread focused on what comes beyond that time. To be true to the phrase "rebooting computing," the systems envisioned by this thread should include a new active device that has circuit behavior distinctly different from a transistor, a new circuit distinctly different from CMOS, or a new device that serves a special role in a non-von Neumann architecture.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
Saturday, March 22, 2014
update of low-power image recognition competition
Committee Members:
These are the current members in the committee. If you want to join (or you want to recommend someone), please send email to me (yunglu@purdue.edu). Thank you.
We are inclined to remove the idea of using a display for giving the test data. This will introduce too much uncertainty in data acquisition through cameras. Instead, we plan to give the data to participants through networks directly.
The competition will be held in mid 2015. The rules will be announced in summer 2014.
These are the current members in the committee. If you want to join (or you want to recommend someone), please send email to me (yunglu@purdue.edu). Thank you.
- Alex Berg (UNC)
- David Kirk (Nvidia)
- Yung-Hsiang Lu (Purdue)
- Gi-Joon Nam (IBM, representative of ACM SIGDA)
We are inclined to remove the idea of using a display for giving the test data. This will introduce too much uncertainty in data acquisition through cameras. Instead, we plan to give the data to participants through networks directly.
The competition will be held in mid 2015. The rules will be announced in summer 2014.
Adiabatic and reversible computing
In an effort to set up a community around adiabatic and reversible computing, we are asking participants in Rebooting Computing that are knowledgeable about this topic to post references to explanatory material.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
We request that people posting technical materials make themselves available to explain the material or to participate in a dialog.
Thursday, March 20, 2014
RCS 2 organizational comments
The following points were discussed by the organizing committee.
Rebooting Computing participants are asked to find future technology options in an expansive way. There is a lot of precedent for identifying technology options that are subjectively or quantitatively better than current technology. However, Rebooting Computing encourages participants to look further. First, the current technology (CMOS, or Moore's Law) will continue to advance for some time. Technology options should be compared against the endpoint of road maps rather than current products. Furthermore, there are multiple radical technologies under consideration. It will be important to see how the forward-looking technologies stack up against each other.
RCS 2 is viewing applications as the driver for technology requirements. The committee is proposing three applications classes:
- Mobile computing
- Learning and reasoning, based on the "executive assistant" from RCS 1
- Servers and supercomputers
- Neuromorphic, or brain inspired
- Incremental improvements on CMOS
- Adiabatic and Reversible
- Approximate computing
(*NLP = Natural Language Processing; *CNT = Carbon Nanotube; *JJ = Josephson Junction)
The organizing committee discussed the following timeline:
Pre-RCS 2: The blog and website will be seeded with information about Rebooting Computing. Also, some participants will be encouraged to create summary information of technology areas. These will provide context for the program at the RCS 2 event.
At RCS 2: Participants will be encouraged to form groups around self-consistent technology ideas. For example, people advocating a hardware technology in conjunction with people interested in a specific aspect of the applications.
Over summer 2014: Participants will be encouraged to maintain a persistent involvement through the website, blog, e-mail, and personal interactions.
At RCS 3, around October 2014: The goal of this event will be to enable participants to offer "rebooted computing" ideas in forum of people that can appreciate the ideas, including contrasting multiple approaches to computing.
Wednesday, January 29, 2014
competition for energy-efficient computing (image recognition)
This figure shows the idea of the competition. The inputs will be a set of images. The score is the ratio of the accuracy and the amount of energy consumed. The participants have the freedom to offload computation.
The competition will be held in 2015 with a major conference (one candidate is IEEE International Symposium on Circuits and Systems in June). The winner must be able to recognize or classify at least half of the images (i.e. the accuracy must exceed 50%).
Prizes:
The competition will be held in 2015 with a major conference (one candidate is IEEE International Symposium on Circuits and Systems in June). The winner must be able to recognize or classify at least half of the images (i.e. the accuracy must exceed 50%).
Prizes:
- first prize is a cash award of USD$5,000
- second prize $3,000
- third $2,000.
Tuesday, January 28, 2014
questions for energy efficiency in Rebooting Computing 2
Hello, Everyone,
The RC (rebooting computing) committee gave me the tasks of writing a
set of questions to discuss for the blogger as well as the next RC
Summit. Here is my first attempt. Please feel free to comment and
change the list. Thank you.
What technologies have the potential replacing CMOS? Where are they
standing now? What are the likely timelines?
Will Boolean logic be replaced? What are the candidates?
Will future computers be hybrid, including some parts that can produce
precise answers (number crunching) and some parts that produce
approximate answers (such as recognition)?
Will it be possible to power most of the "edge devices" (sensors,
mobile phones, tablets) by ambient energy sources? If so, how? If
not, why?
What is the theoretical minimum energy for computing, communication,
and storage?
Let me further define the units of computing, communication,
and storage.
One unit for computing is one double-precision floating-point
division.
One unit of communication is sending 1MB and receiving 1MB data
(excluding headers) for wireless: 10 meters away; for wires or fiber:
100 meters away.
One unit of storage is saving 1MB data (without metadata).
What is the state of the art in the energy consumption for computing,
communication, and storage?
What is the distance between the theoretically minimum and the current
state of the art?
What are the barriers reducing the distances?
How are different power management solutions evaluated? Do you think
future electronics should have "energy counters" that can report their
energy consumption? Would this accelerate the research and innovation
improving energy efficiency?
Is it possible to develop "energy benchmarks" (similar to performance
benchmarks)? As most computing devices are connected to networks,
would it be necessary to develop energy benchmarks that consider
network connections? How should the energy consumed by shared
resources (routers and servers) be counted?
The RC (rebooting computing) committee gave me the tasks of writing a
set of questions to discuss for the blogger as well as the next RC
Summit. Here is my first attempt. Please feel free to comment and
change the list. Thank you.
What technologies have the potential replacing CMOS? Where are they
standing now? What are the likely timelines?
Will Boolean logic be replaced? What are the candidates?
Will future computers be hybrid, including some parts that can produce
precise answers (number crunching) and some parts that produce
approximate answers (such as recognition)?
Will it be possible to power most of the "edge devices" (sensors,
mobile phones, tablets) by ambient energy sources? If so, how? If
not, why?
What is the theoretical minimum energy for computing, communication,
and storage?
Let me further define the units of computing, communication,
and storage.
One unit for computing is one double-precision floating-point
division.
One unit of communication is sending 1MB and receiving 1MB data
(excluding headers) for wireless: 10 meters away; for wires or fiber:
100 meters away.
One unit of storage is saving 1MB data (without metadata).
What is the state of the art in the energy consumption for computing,
communication, and storage?
What is the distance between the theoretically minimum and the current
state of the art?
What are the barriers reducing the distances?
How are different power management solutions evaluated? Do you think
future electronics should have "energy counters" that can report their
energy consumption? Would this accelerate the research and innovation
improving energy efficiency?
Is it possible to develop "energy benchmarks" (similar to performance
benchmarks)? As most computing devices are connected to networks,
would it be necessary to develop energy benchmarks that consider
network connections? How should the energy consumed by shared
resources (routers and servers) be counted?
Thursday, January 16, 2014
Action Items for energy efficiency
From the summit, I think there are two main actions items for the energy efficiency group.
- Organize a competition of low-power designs.
- Organize workshops discussing how to design systems that can migrate computation to more energy efficient locations or locations with renewable energy
Subscribe to:
Posts (Atom)