Milkymist
Encyclopedia
The Milkymist project is an informal organization
of people and companies who develop, manufacture and sell a comprehensive open source
solution for the live synthesis
of interactive visual effects for VJs
. The project goes great lengths to apply the open source principles at every level possible, and is best known for the Milkymist system-on-chip (SoC) which is among the first commercialized system-on-chip designs with free
HDL
source code.
As a result, several Milkymist technologies have been reused in applications unrelated to video synthesis. For example, NASA
's Communication Navigation and Networking Reconfigurable Testbed (CoNNeCT) experiment uses the memory controller
that was originally developed for the Milkymist system-on-chip and published under the GNU GPL.
The project was presented at several open source and hacker
conferences such as the Chaos Communication Congress
, FOSDEM
, Libre Software Meeting
, and Libre Graphics Meeting 2011
. It was also featured on the Make Magazine blog and included in their "ultimate open source hardware gift guide 2010".
(LM32) core as a general purpose processor. It is a RISC 32-bit big endian CPU without a memory management unit
(MMU). It is supported by the GCC
compiler and can run RTEMS
and uClinux
. There is also an experimental back-end for LLVM targeting this microprocessor.
The LM32 microprocessor is assisted by a texture mapping unit
and a programmable floating point
VLIW coprocessor which are used by the Flickernoise video synthesis software. It is also surrounded by various peripheral cores to support every I/O device of the Milkymist One. The system-on-chip interconnect uses three bridged buses and mixes the Wishbone
protocol with two custom protocols used for configuration registers and high performance DMA
with the SDRAM
.
The architecture of the Milkymist system-on-chip is largely documented in the project founder's Master thesis report. Most components of the system-on-chip, except the LatticeMico32
core, were custom developed and placed under the GNU GPL license.
The QEMU
emulator can be used to run and debug Milkymist SoC binaries on another computer.
and reconfigurable computer
is the main product released by the project. It is manufactured by Qi Hardware
, a start-up founded by former Openmoko
employees. It was first sold at the Chaos Communication Congress
in 2010, as an "early developer kit" for interested hackers, open source
activists and pioneers who could stand the remaining software and FPGA design bugs. The hardware still came with a warranty against manufacturing defects. Moreover, the Milkymist project has plans to level the Milkymist One up to a general (non-technical) public device marketed at clubs, music bands and VJs
.
The technical specifications of the Milkymist One are as follows:
Following the open source principles that guide the project, the design files of the printed circuit board
and the CAD files of the case were released under the Creative Commons
Attribution-Share Alike license.
and uses a similar, and largely compatible, scripting language to define and program the visual effects. However, while MilkDrop is designed to run automatically in a music player, Flickernoise puts the accent on the interactivity of the visuals and live performance uses. The software supports the programming of visual effects that transform a live video stream coming from a camera connected to the Milkymist One, as well as input from OpenSoundControl, DMX512 and MIDI controllers.
Flickernoise runs on the RTEMS
real-time operating system, and uses many POSIX
software libraries that were ported to this operating system such as libpng
, libjpeg
, jbig2dec, openjpeg
, Freetype
, MuPDF
and liblo for OpenSoundControl support. The streamlined hardware platform along with the use of a real-time operating system allows the system to have a lower response time than an equivalent PC based setup. The user interface is based on a variant of the Genode FX toolkit.
Flickernoise is also free software
, released under the GNU GPL.
Informal organization
The informal organization is the interlocking social structure that governs how people work together in practice. It is the aggregate of behaviors, interactions, norms, personal and professional connections through which work gets done and relationships are built among people who share a common...
of people and companies who develop, manufacture and sell a comprehensive open source
Open source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
solution for the live synthesis
Video synthesizer
A Video Synthesizer is a device that electronically creates a video signal.A video synthesizer is able to generate a variety of visual material without camera input through the use of internal video pattern generators, as seen in the stillframes of motion sequences shown above. It can also accept...
of interactive visual effects for VJs
VJ (video performance artist)
VJing is a broad designation for realtime visual performance. Characteristics of VJing are the creation or manipulation of imagery in realtime through technological mediation and for an audience, in synchronization to music. VJing often takes place at events such as concerts, nightclubs, music...
. The project goes great lengths to apply the open source principles at every level possible, and is best known for the Milkymist system-on-chip (SoC) which is among the first commercialized system-on-chip designs with free
Free software
Free software, software libre or libre software is software that can be used, studied, and modified without restriction, and which can be copied and redistributed in modified or unmodified form either without restriction, or with restrictions that only ensure that further recipients can also do...
HDL
Hardware description language
In electronics, a hardware description language or HDL is any language from a class of computer languages, specification languages, or modeling languages for formal description and design of electronic circuits, and most-commonly, digital logic...
source code.
As a result, several Milkymist technologies have been reused in applications unrelated to video synthesis. For example, NASA
NASA
The National Aeronautics and Space Administration is the agency of the United States government that is responsible for the nation's civilian space program and for aeronautics and aerospace research...
's Communication Navigation and Networking Reconfigurable Testbed (CoNNeCT) experiment uses the memory controller
Memory controller
The memory controller is a digital circuit which manages the flow of data going to and from the main memory. It can be a separate chip or integrated into another chip, such as on the die of a microprocessor...
that was originally developed for the Milkymist system-on-chip and published under the GNU GPL.
The project was presented at several open source and hacker
Hacker (computer security)
In computer security and everyday language, a hacker is someone who breaks into computers and computer networks. Hackers may be motivated by a multitude of reasons, including profit, protest, or because of the challenge...
conferences such as the Chaos Communication Congress
Chaos Communication Congress
The Chaos Communication Congress is an annual meeting of the international hacker scene, organized by the Chaos Computer Club. The congress features a variety of lectures and workshops on technical and political issues....
, FOSDEM
FOSDEM
FOSDEM is a non-commercial volunteer organized European event centered around free and open source software development. It is aimed at developers and anyone interested in the free and open source software movement...
, Libre Software Meeting
Libre Software Meeting
The Libre Software Meeting is an annual free software event in France. The French name of this event is Rencontres mondiales du logiciel libre .- Summary :...
, and Libre Graphics Meeting 2011
Libre Graphics Meeting
The Libre Graphics Meeting is an annual convention for the discussion of free and open source software used with graphics; The first Libre Graphics Meeting was held in May 2006. Communities from Inkscape, GIMP, Krita, Scribus, sK1, Blender, Open Clip Art Library, Open Font Library, and more come...
. It was also featured on the Make Magazine blog and included in their "ultimate open source hardware gift guide 2010".
Milkymist SoC
The Milkymist system-on-chip uses the LatticeMico32LatticeMico32
LatticeMico32 is a 32-bit microprocessor soft core from Lattice Semiconductor optimized for field-programmable gate arrays . It uses a Harvard architecture, which means the instruction and data buses are separate. Bus arbitration logic can be used to combine the two buses, if desired.LatticeMico32...
(LM32) core as a general purpose processor. It is a RISC 32-bit big endian CPU without a memory management unit
Memory management unit
A memory management unit , sometimes called paged memory management unit , is a computer hardware component responsible for handling accesses to memory requested by the CPU...
(MMU). It is supported by the GCC
GNU Compiler Collection
The GNU Compiler Collection is a compiler system produced by the GNU Project supporting various programming languages. GCC is a key component of the GNU toolchain...
compiler and can run RTEMS
RTEMS
RTEMS is a free open source real-time operating system designed for embedded systems....
and uClinux
UClinux
μClinux stands for "MicroController Linux," and is pronounced "you-see-Linux" as explained on the website, not the way the Greek letter mu is normally pronounced. It was a fork of the Linux kernel for microcontrollers without a memory management unit...
. There is also an experimental back-end for LLVM targeting this microprocessor.
The LM32 microprocessor is assisted by a texture mapping unit
Texture mapping unit
A texture mapping unit is a component in modern graphics processing units , historically it is a separate physical processor. A TMU is able to rotate and resize a bitmap to be placed onto an arbitrary plane of a given 3D object as a texture...
and a programmable floating point
Floating point
In computing, floating point describes a method of representing real numbers in a way that can support a wide range of values. Numbers are, in general, represented approximately to a fixed number of significant digits and scaled using an exponent. The base for the scaling is normally 2, 10 or 16...
VLIW coprocessor which are used by the Flickernoise video synthesis software. It is also surrounded by various peripheral cores to support every I/O device of the Milkymist One. The system-on-chip interconnect uses three bridged buses and mixes the Wishbone
Wishbone (computer bus)
The Wishbone Bus is an open source hardware computer bus intended to let the parts of an integrated circuit communicate with each other. The aim is to allow the connection of differing cores to each other inside of a chip...
protocol with two custom protocols used for configuration registers and high performance DMA
DMA
DMA can refer to:* DMA , a defunct dance music magazine* Dallas Museum of Art, an art museum in Texas, USA* Danish Music Awards, an award show held in Denmark since 1989...
with the SDRAM
SDRAM
Synchronous dynamic random access memory is dynamic random access memory that is synchronized with the system bus. Classic DRAM has an asynchronous interface, which means that it responds as quickly as possible to changes in control inputs...
.
The architecture of the Milkymist system-on-chip is largely documented in the project founder's Master thesis report. Most components of the system-on-chip, except the LatticeMico32
LatticeMico32
LatticeMico32 is a 32-bit microprocessor soft core from Lattice Semiconductor optimized for field-programmable gate arrays . It uses a Harvard architecture, which means the instruction and data buses are separate. Bus arbitration logic can be used to combine the two buses, if desired.LatticeMico32...
core, were custom developed and placed under the GNU GPL license.
The QEMU
QEMU
QEMU is a processor emulator that relies on dynamic binary translation to achieve a reasonable speed while being easy to port on new host CPU architectures....
emulator can be used to run and debug Milkymist SoC binaries on another computer.
Milkymist One
The Milkymist One video synthesizerVideo synthesizer
A Video Synthesizer is a device that electronically creates a video signal.A video synthesizer is able to generate a variety of visual material without camera input through the use of internal video pattern generators, as seen in the stillframes of motion sequences shown above. It can also accept...
and reconfigurable computer
Reconfigurable computing
Reconfigurable computing is a computer architecture combining some of the flexibility of software with the high performance of hardware by processing with very flexible high speed computing fabrics like field-programmable gate arrays...
is the main product released by the project. It is manufactured by Qi Hardware
Qi hardware
Qi hardware, also called Copyleft hardware, is hardware that attempts to apply the Free Software Foundation's GNU GPL concept of copylefting software to the hardware layer. The project is both a popular open hardware community websites and a company, co-founded by Wolfgang Spraul and Yi Zhang, that...
, a start-up founded by former Openmoko
Openmoko
Openmoko is a project to create a family of open source mobile phones, including the hardware specification and the operating system. The project was sponsored by Openmoko Inc....
employees. It was first sold at the Chaos Communication Congress
Chaos Communication Congress
The Chaos Communication Congress is an annual meeting of the international hacker scene, organized by the Chaos Computer Club. The congress features a variety of lectures and workshops on technical and political issues....
in 2010, as an "early developer kit" for interested hackers, open source
Open source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
activists and pioneers who could stand the remaining software and FPGA design bugs. The hardware still came with a warranty against manufacturing defects. Moreover, the Milkymist project has plans to level the Milkymist One up to a general (non-technical) public device marketed at clubs, music bands and VJs
VJ (video performance artist)
VJing is a broad designation for realtime visual performance. Characteristics of VJing are the creation or manipulation of imagery in realtime through technological mediation and for an audience, in synchronization to music. VJing often takes place at events such as concerts, nightclubs, music...
.
The technical specifications of the Milkymist One are as follows:
- Multi-standard video input (PALPALPAL, short for Phase Alternating Line, is an analogue television colour encoding system used in broadcast television systems in many countries. Other common analogue television systems are NTSC and SECAM. This page primarily discusses the PAL colour encoding system...
/SECAMSECAMSECAM, also written SÉCAM , is an analog color television system first used in France....
/NTSCNTSCNTSC, named for the National Television System Committee, is the analog television system that is used in most of North America, most of South America , Burma, South Korea, Taiwan, Japan, the Philippines, and some Pacific island nations and territories .Most countries using the NTSC standard, as...
) - Two DMX512 (RS485) ports
- MIDI IN and MIDI OUT ports
- SVGA output, 24bpp, up to 140 MHz pixel clock (about 1280x1024)
- AC97AC97AC'97 is Intel Corporation's Audio Codec standard developed by the Intel Architecture Labs in 1997, and used mainly in motherboards, modems, and sound cards....
audio - XC6SLX45 Spartan-6 FPGA supporting the open source Milkymist SoC
- 128MB 32-bit DDR400 SDRAMSDRAMSynchronous dynamic random access memory is dynamic random access memory that is synchronized with the system bus. Classic DRAM has an asynchronous interface, which means that it responds as quickly as possible to changes in control inputs...
- 32MB parallel flash
- 10/100 EthernetEthernetEthernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....
- Memory cardMemory cardA memory card or flash card is an electronic flash memory data storage device used for storing digital information. They are commonly used in many electronic devices, including digital cameras, mobile phones, laptop computers, MP3 players, and video game consoles...
- Two USB host connectors
- RC-5RC-5The RC-5 protocol was developed by Philips in the late 1980s as a semi-proprietary consumer IR remote control communication protocol for consumer electronics. However, it was also adopted by most European manufacturers, as well as many US manufacturers of specialty audio and video equipment...
compatible infrared receiver - RS232 debug port
Following the open source principles that guide the project, the design files of the printed circuit board
Printed circuit board
A printed circuit board, or PCB, is used to mechanically support and electrically connect electronic components using conductive pathways, tracks or signal traces etched from copper sheets laminated onto a non-conductive substrate. It is also referred to as printed wiring board or etched wiring...
and the CAD files of the case were released under the Creative Commons
Creative Commons
Creative Commons is a non-profit organization headquartered in Mountain View, California, United States devoted to expanding the range of creative works available for others to build upon legally and to share. The organization has released several copyright-licenses known as Creative Commons...
Attribution-Share Alike license.
Flickernoise
Flickernoise is the video synthesis software that runs on the Milkymist One. It is heavily inspired by MilkDropMilkDrop
MilkDrop is an open source, hardware-accelerated music visualization plugin for Winamp, which was originally developed by Ryan Geiss. It uses DirectX and intelligent beat detection to render iterated images which blend seamlessly...
and uses a similar, and largely compatible, scripting language to define and program the visual effects. However, while MilkDrop is designed to run automatically in a music player, Flickernoise puts the accent on the interactivity of the visuals and live performance uses. The software supports the programming of visual effects that transform a live video stream coming from a camera connected to the Milkymist One, as well as input from OpenSoundControl, DMX512 and MIDI controllers.
Flickernoise runs on the RTEMS
RTEMS
RTEMS is a free open source real-time operating system designed for embedded systems....
real-time operating system, and uses many POSIX
POSIX
POSIX , an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems...
software libraries that were ported to this operating system such as libpng
Libpng
libpng is the official Portable Network Graphics reference library . It is a platform-independent library that contains C functions for handling PNG images...
, libjpeg
Libjpeg
libjpeg is a library written entirely in C which contains a widely-used implementation of a JPEG decoder, JPEG encoder and other JPEG utilities...
, jbig2dec, openjpeg
OpenJPEG
OpenJPEG is an open-source library to encode and decode JPEG 2000 images.-Support for lossless 16-bit images:Unlike JasPer, another open-source JPEG 2000 implementation, OpenJPEG fully respects the JPEG 2000 specification and can compress/decompress lossless 16-bit images.-Origin:OpenJPEG is a fork...
, Freetype
FreeType
FreeType is a software library written in C that implements a font rasterization engine. It is used to render text on to bitmaps and provides support for other font-related operations.-Details:...
, MuPDF
MuPDF
MuPDF is an open source software library written in C that implements a PDF and XPS parsing and rendering engine. It is used primarily to render pages into bitmaps, but also provides support for other operations such as searching and listing the table of contents and hyperlinks.The focus of MuPDF...
and liblo for OpenSoundControl support. The streamlined hardware platform along with the use of a real-time operating system allows the system to have a lower response time than an equivalent PC based setup. The user interface is based on a variant of the Genode FX toolkit.
Flickernoise is also free software
Free software
Free software, software libre or libre software is software that can be used, studied, and modified without restriction, and which can be copied and redistributed in modified or unmodified form either without restriction, or with restrictions that only ensure that further recipients can also do...
, released under the GNU GPL.