OVERVIEW
TIME |
MONDAY 12/10 |
Session Chair |
|
8:00
– 8:30 |
Coffee
& Bagel |
|
|
8:30
– 9:10 |
Inv.
Talk 1: Andrew Viterbi |
Terrence Sejnowski |
|
9:10
– 10:30 |
Session
1: Theory & Algorithms I |
Erkki
Oja |
|
10:30
– 10:50 |
Coffee
break |
|
|
10:50
– 12:30 |
Session
2: Applications I |
Luis
Almeida |
|
12:30
– 2:00 |
Lunch |
|
|
2:00
– 2:40 |
Inv.
Talk 2: Mohan Trivedi |
Te-Won
Lee |
|
2:40
– 4:00 |
Session
3: Theory & Algorithms II |
Lars
Kai Hansen |
|
4:00
– 4:20 |
Tea
break |
|
|
4:20
– 5:50 |
Poster
1 |
Eric
Moreau, Tzyy-Ping Jung |
|
5:50
– 6:30 |
|
|
|
7:00
– 8:00 |
Social
Event & Snacks |
|
|
8:00
– 9:00 |
Poster
2 |
Seungjin
Choi, Te-Won Lee |
TIME |
TUESDAY 12/11 |
Session Chair |
|
8:00
– 8:30 |
Coffee
& Bagel |
|
|
8:30
– 9:10 |
Inv.
Talk 3: Michael Jordan |
Tony Bell |
|
9:10
– 10:30 |
Session
4: Biomedical Signal Processing |
Scott
Makeig |
|
10:30
– 10:50 |
Coffee
break |
|
|
10:50
– 12:30 |
Session
5: Theory & Algorithms III |
J.F.
Cardoso |
|
12:30
– 2:00 |
Lunch |
|
|
2:00
– 2:40 |
Inv.
Talk 4: Gilles Laurent |
Christian
Jutten |
|
2:40
– 4:00 |
Session
6: Applications II |
Scott
Douglas |
|
4:00
– 4:20 |
Tea
break |
|
|
4:20
– 5:50 |
Poster
3 |
Tzyy-Ping
Jung, Te-Won Lee |
|
5:50
– 6:30 |
Inv.
Talk 5: Robert Hecht-Nielsen |
Javier
Movellan |
|
7:00
– 8:00 |
Banquet
Dinner |
|
|
8:00
– 9:00 |
Tony
Bell
Banquet Talk |
Terrence
Sejnowski |
TIME |
WEDNESDAY 12/12 |
Session Chair |
|
8:00
– 8:30 |
Coffee
& Bagel |
|
|
8:30
– 9:10 |
Inv.
Talk 6: Bhaskar Rao |
Te-Won Lee |
|
9:10
– 10:30 |
Session
7: Convolved Sources |
Andrzej
Cichocki |
|
10:30
– 10:50 |
Coffee
break |
|
|
10:50
– 12:30 |
Session
8: Nonlinear Representation |
Juha
Karhunen |
|
12:30
– 2:00 |
Lunch (Salk)
|
|
|
2:00
– 2:40 |
Inv.
Talk 7: Christof Koch |
Barak
Pearlmutter |
|
2:40
– 4:00 |
Session
9: Speech Signal Processing |
Lars
Kai Hansen |
|
4:00
– 4:20 |
Tea
break |
|
|
4:20
– 5:50 |
Inv.
Talk 8: Geoffrey Hinton |
Terrence
Sejnowski |
|
5:50
– 6:00 |
Closing
remarks |
|
MONDAY, DECEMBER 10
8:00 – 8:15 Coffee & Bagel
8:15
– 8:30 Opening Remarks: Terrence Sejnowski (CNL, The Salk
Institute,
INC, UCSD)
|
8:30 – 9:10 Inv.
Talk 1-- Andrew Viterbi (Qualcomm, Inc) Low-Density Parity Check Codes: Approaching the Last Frontier of Shannon Theory
|
|
9:10 – 10:30 Session
1: Theory & Algorithms I 9:10 – 9:30 Jean-Francois Cardoso [134] The Three Easy Routes to Independent Component Analysis; Contrasts and Geometry
9:30 – 9:50 Deniz Erdogmus, José C. Príncipe [56]
9:50 – 10:10 Riccardo Boscolo, Hong Pan, Vwani P. Roychowdhury [87] |
|
10:10 -10:30 Spotlights Noboru Murata [45] Properties of the Empirical Characteristic Function and it Application to Testing for Independence Vincente Zarzoso, Asoke K. Nandi [37] A General Theory of Closed-Form Estimators for Blind Source Separation
Yasu Matsuyama, Naoto Katsumata, Shuichiro Imahara [64] Convex Divergence as a Surrogate Function for Independence: The f-Divergence ICA Dinh Tuan Pham [58] Contrast Functions for Blind Separation and |
10:30 – 10:50 Coffee Break
|
10:50 – 12:30 Session
2: Applications I 10:50 - 11:10 Maria Funaro, Erkki Oja, Harri Valpola [4] Artifact Detection in Astrophysical Image Data Using Independent Component Analysis
11:10 – 11:30 Milutin Stanacevic, Gert Cauwenberghs, George Zweig [90b] Gradient Flow Broadband Beamforming and Source Separation
11:30 – 11:50 Tapani Ristaniemi, Rui Wu [132] Mitigation of Continuous-Wave Jamming in DS-CDMA Systems Using Blind Source Separation Techniques 11:50 – 12:10 Xiaoan Sun, Scott C. Douglas [128] A Natural Gradient Convolutive Blind Source Separation |
|
12:10 – 12:30 Spotlights Wolf Baumann, Bert-Uwe
Kohler, Dorothea Kolossa and Reinhold Orglmeister [73] Real Time Separation of Convolutive Mixtures Ki-Seok Cho, Soo-Young Lee [20] Implementation of INFOMAX ICA Algorithm with Analog
Frank Meinecke, Andreas Zieche, Motoaki Kawanabe, Klaus R.Mueller [112] Assessing Reliability of ICA Projections – A Resampling Harold Szu [141] A Priori Maxent H(S) Independent Class Analysis (ICA) Vs. A Posteriori Maxent H(V) ICA |
12:30 – 2:00 Lunch
|
2:00 – 2:40 Inv.
Talk 2 -- Mohan Trivedi (UCSD) "Visual Networks for Intelligent Spaces" |
|
2:40 – 4:00 Session 3: Theory & Algorithms II 2:40 -3:00 R.A. Choudrey and S.J. Roberts [47] Flexible Bayesian Independent Component Analysis for Blind Source Separation
3:00 – 3:20 Albert Bijaoui, Danielle Nuzillard Smoothing and Adaptive Denoising for Blind Source Separation
3:20 – 3:40 Stefan Harmeling, Andreas Ziehe, Motoaki Kawanabe, Benjamin Blankertz, Klaus-Robert Muller [80] |
|
3:40 – 4:00 Spotlights J.
Eriksson, A. Kankainen, V. Koivunen [42] Novel Characteristic Function Based Criteria for ICA Toshinao Akuzawa [68] New Fast Factorization Method for Multivariate Optimization and its Realization as ICA Algorithm
Wei Lu, Jagath C. Rajapakse [70]
Kenneth E. Hild II, Deniz Erdogmus,
Jose C. Principe [130] On-Line Minimum Mutual Information Method for Time-Varying Blind Source Separation |
4:00 – 4:20 Tea
Break
7:00 – 8:00 Social
Event & Food
TUESDAY, DECEMBER 11
8:00 – 8:30 Coffee & Bagel
|
8:30 – 9:10 Inv.
Talk 3 -- Michael Jordan (UC
Berkeley) Kernel Independent Component Analysis |
|
9:10 – 10:30 Session
4: Biomedical Signal Processing 9:10 – 9:30 Pedro Hojen-Sorensen, Lars Kai Hansen, Ole Winther [124] Mean Field Implementation of Bayesian ICA 9:30 – 9:50 Jianting Cao, Noboru Murata, Shunichi Amari, Andrzejo Cichocki,Tsunehiro Takeda [19] A Robust ICA Approach for Unaveraged Single-Trial Auditory Evoked Fields Data Decomposition 9:50 – 10:10 Jaakko Särelä, Harri Valpola, Ricardo Vigário, and Erkki Oja [44] Dynamical Factor Analysis of Rhythmic Magnetoencephalographic Activity
|
|
10:10 – 10:30 Spotlights A.Delorme, S. Makeig, T. Sejnowski [117] Automatic Artifact Rejection in EEG Using High Order Statistics and Independent Component Analysis Kevin H. Knuth, Wilson A. Truccolo, Steven L. Bressler, Mingzhou Ding [100] Separation of Multiple Evoked Responses using Differential Amplitude and Latency Variability Jagath C. Rajapakse, Wei Lu [63] Extracting Task-Related Components in Functional MRI Irina F. Gorodnitsky , Adel Belouchrani [2] Joint Cumulant and Correlation Based Signal Separation with Application to EEG Data Analysis
|
10:30 –
10:50 Coffee Break
|
10:50 – 12:30 Session
5: Theory & Algorithms III 10:50 – 11:10 Tim Marks, Javier R. Movellan [21] Diffusion Networks, Product of Experts, and 11:10 - 11:30 Laurent Giulieri, Nadège Thirion-Moreau and Pierre-Yves Arquès [10] Blind Source Separation Using Bilinear and Quadratic Time- Frequency Representations 11:30 - 11:50 Kwokleung Chan, TeWon Lee, Terrence Sejnowski [133] Variational Learning of Clusters of Undercomplete Nonsymmetric Independent Components 11:50 – 12:10 Sepp Hochreiter, Michael C. Mozer [131] Monaural Separation and Classification of Mixed Signals: A Support-Vector Regression Perspective
|
|
12:10 – 12:30 Spotlights Pavel Kisilev,
Michael Zibulevsky, Yehoshua Y. Zeevi, Barak A. Pearlmutter [54] Blind Source
Separation via Multinode Sparse
Juan K. Lin [104] Lattice Decompositions of Multivariate Probability Density Functions Extending the ICA Model to Incorporate More General Source Structures
Mark Plumbley [94] Adaptive Lateral Inhibition for Non-Negative ICA Arie Yeredor [121] Blind Source Separation with Pure Delay Mixtures |
12:30 –
2:00 Lunch
|
2:00 – 2:40 Inv.
Talk 4 -- Gilles Laurent
(Caltech) Dynamics and computation in olfactory circuit |
|
2:40 – 4:00 Session
6: Applications II 2:40 – 3:00 J.-R. Duann, T.-P. Jung, W.-J. Kuo, T.-C. Yeh, S. Makeig, J.-C. Hsieh, T. J. Sejnowski [140] Measuring the Variability of Event-Related Bold Signal 3:00 – 3:20 Samer A. Abdallah, Mark D. Plumbley [52] If the Independent Components of Natural Images are Edges, What are the Independent Components of Natural 3:20 – 3:40 Thomas Kolenda, Lars Kai Hansen, Jan Larsen [76] Signal Detection Using ICA: Application to Chat |
|
3:40 – 4:00 Spotlights Ella Bingham
[39] Topic Identification in Dynamical Text by Extracting Minimum Complexity Time Components Johan Himberg, Aapo Hyvaerinen [43] Independent Component Analysis for Binary Yu-Hwan Kim, Byoung-Tak Zhang [135] Document Indexing Using Independent Topic Extraction Wakako Hashimoto [72] Independent Component Analysis with Several |
4:00 – 4:20 Tea Break
|
5:50 – 6:30 Inv.
Talk 5 -- Robert Hecht Nielsen
(HNC, Inc.) A
Real-Time Language-Knowledge-Dependent Solution |
7:00 – 8:00 Banquet
Dinner
|
8:00 – 9:00 Tony
Bell Banquet Talk -- (Salk, UCSF) The Generative Model |
WEDNESDAY,
DECEMBER 12
8:00 – 8:30 Coffee & Bagel
|
8:30 – 9:10 Inv. Talk 6 -- Bhaskar Rao Algorithms for Computing Sparse Solutions |
|
9:10 – 10:30 Session
7: Convolved Sources 9:10 – 9:30 Pierre Comon, Eric Moreau, Ludwig Rota [29] Blind Separation of Convolutive Mixtures: A Contrast-Based Diagonalization Approach 9:30 – 9:50 M. Kawamoto, Y. Inouye, A. Mansour, and R.W. Liu [92] Blind Deconvolution Algorithms for MIMO-FIR Systems Driven by Fourth-Order Colored Signals 9:50 – 10:10 Naoki Saito, Bertrand Benichou [46] The Spike Process: A Simple Test Case for Independent
|
|
10:10 – 10:30 Spotlights N. Mitianoudis,
M. Davies [25] New Fixed-Point ICA Algorithms for Convolved Mixtures Gil-Jin Jang, Te-Won Lee, Yung-Hwan Oh [109] Blind Separation of Single Channel Mixture Using Inseon Jang, Seungjin Choi [81] Sequential Least Squares Algorithms for Blind Shoko Araki,Shoji Makino ,Ryo Mukai, Tsuyoki Nishikawa, Hiroshi Saruwatari [35] Fundamental Limitation of Frequency Domain Blind Source Separation for Convolved Mixture of Speech |
10:30 – 10:50 Coffee Break
|
10:50 – 12:30 Session 8: Nonlinear
Representation 10:50 – 11:10 Alexandre Iline, Harri Valpola, Erkki Oja [12] Detecting Process State Changes by Nonlinear Blind
11:10 – 11:30 Harri Valpola, Tapani Raiko, Juha Karhunen [107] Building Blocks for Hierarchical Latent Variable Models 11:30 – 11:50 Torbjorn Eltoft, Orjan Kristiansen [51] ICA and Nonlinear Time Series Prediction for Recovering Missing Data Segments in Multivariate Signals 11:50 – 12:10 Kiyotoshi Matsuoka, Satoshi Nakashima [99] Minimal Distortion Principle for Blind Source Separation
|
|
12:10 – 12:30 Spotlights Sophie Achard,
Dinh Tuan Pham, Christian Jutten [50] Blind Source Separation in Post Nonlinear Mixtures Douglas R. Hundley, Michael J. Kirby, Markus G. Anderle [115] A Solution Procedure for Blind Signal Separation Using the Maximum Noise Fraction Approach: Andreas Ziehe, Motoaki Kawanabe, Stefan Harmeling, Klaus-Robert Muller [120] Separation of Post-Nonlinear Mixtures Using Ace and Russell H. Lambert, Marcel Joho, Heinz Mathis [85] Polynomial Singular Values for Number of Wideband Sources Estimation and Principal Component Analysis
|
12:30 –
2:00 Lunch (at the Salk Institute)
|
2:00 – 2:40 Inv.
Talk 7 -- Christof Koch (Caltech) Responses of Single Neurons in The Human Medial Temporal Lobe During Visual Simulation, Imagery And Flash Suppression |
|
2:40 – 4:00
Session 9:
Speech Signal Processing 2:40 – 3:00 Tomasz Rutkowski, Andrzej Cichocki, Allan Kardec Barros [86] Speech Enhancement from Interfering Sounds Using CASA Techniques and Blind Source Separation 3:00 – 3:20 Frederic Abrard, Yannick Deville, Paul White [53] From Blind Source Separation to Blind Source Cancellation In the Underdetermined Case: A New Approach Based on Time-Frequency Analysis 3:20 – 3:40 Sergio Cruces, Andrzej Cichocki, Shun-ichi Amari [82] Criteria for the Simultaneous Blind Extraction of |
|
3:40 – 4:00 Spotlights Scott Rickard,
Radu Balan, Justinian Rosca [84] Real-Time Time-Frequency Based Blind Source Andrzej Cichocki, Adel Belouchrani [3] Source Separation of Temporally Correlated Source Using Bank of Band Pass Filters Kamran Rahbar, James P. Reilly [55] Blind Source Separation Algorithm for MIMO Radu Balan, Justinian Rosca, Scott Rickard [88] Robustness of Parametric Source Demixing |
4:00 – 4:20 Tea
Break
|
4:20 – 5:30 Inv.
Talk 8 -- Geoffrey Hinton
(University of Toronto) |
5:30 Closing remarks
Inv. Talk-Abstract Section
Andrew Viterbi
(Qualcomm, Inc.)
Low-Density Parity Check Codes: Approaching the Last Frontier of Shannon Theory
For over half a century, information theorists and communication engineers have been refining methods to transmit reliably over noisy channels at rates ever closer to the Shannon Capacity limit. The class of low-density parity check (LDPC) codes, originally proposed by Gallager (1963), have been shown within the last year to be capable of approaching within epsilon of this elusive goal.
We describe the nature of the codes and particularly the iterative decoding algorithms which make this performance possible. We then demonstrate examples of the analysis, known as density evolution, which establishes the decoder’s convergence to the correct estimate.
Dr. Andrew Viterbi is a co-founder and retired Vice Chairman and Chief Technical Officer of QUALCOMM Incorporated. He spent equal portions of his career in industry, having also co-founded a previous company, and in academia as Professor in the Schools of Engineering first at UCLA and then at UCSD, at which he is now Professor Emeritus. He is currently president of the Viterbi Group, a technical advisory and investment company.
His principal
research contribution, the Viterbi Algorithm, is used in most digital cellular
phones and digital satellite receivers, as well as in such diverse fields as
magnetic recording, voice recognition and DNA sequence analysis. In recent years he has concentrated his
efforts on establishing CDMA as the multiple access technology of choice for
cellular telephony and wireless data communication.
Dr. Viterbi has received numerous honors both in the U.S. and internationally. Among these are four honorary doctorates as well as memberships in the National Academy of Engineering, the National Academy of Sciences and the American Academy of Arts and Sciences. He has received the Marconi International Fellowship Award, the IEEE Alexander Graham Bell and Claude Shannon Awards , the NEC C&C Award, the Eduard Rhein Award and the Christopher Columbus Medal.
Mohan Trivedi (University of California at San Diego)
Visual Networks for Intelligent Spaces
Awareness of the space and activities taking place in them are essential requirements of Intelligent Spaces. In this presentation we will describe research efforts directed towards the realization of such intelligent spaces, by using visual and audio sensor networks imbedded in the environment to make them aware of the space and activities taking place in them. The visual awareness is derived utilizing multiple video streams acquired from both the traditional rectilinear as well as the omni directional sensor arrays. This "active" vision system systematically processes the signal-, focus of attention-, and context- level
information. This allows the spaces to derive and continuously maintain an awareness of the identities, locations, and activities of various entities (e.g. people, vehicles, or avatars) inhabiting these spaces. We will present results of our recent research involving "indoor", outdoor and mobile intelligent spaces. Hopefully, these efforts will underline the important role that robust feature extraction and decision theoretical frameworks, like ICA, can play as well as the exciting opportunities in the development of a wide range of intelligent (or "smart") systems.
Mohan M. Trivedi is a Professor in the Electrical and Computer Engineering
Department of the University of California, San Diego where he serves as the
Director of the Computer Vision and Robotics Research Laboratory (http://cvrr.ucsd.edu). He and his team are engaged in a broad range of research studies
in active perception and novel machine vision systems, intelligent (“smart”)
environments, distributed video networks and intelligent systems. At UCSD
Trivedi also serves on the Executive Committee of the California Institute
for Telecommunication and Information Technologies, Cal-(IT)2,
leading the team involved in the Intelligent Transportation and Telematics
project. He also serves as a charter member of the Executive Committee of the
University of California System-wide Digital Media Innovation (http://www.dimi.ucsb.edu/) (DiMI) Program. Trivedi serves as
the Editor-in-Chief of Machine Vision and Applications, the official journal of
the International Association of Pattern Recognition. He has published extensively and has edited over a dozen volumes
including books, special issues, video presentations, and conference
proceedings. He servers regularly as a
consultant to various national and international industry and government
agencies. Trivedi is a recipient of the
Pioneer Award (Technical Activities) and the Meritorious Service Award of the
IEEE Computer Society and the Distinguished Alumnus Award from the Utah State
University. He is a Fellow of the International Society for Optical Engineering
(SPIE). He has been elected to the honor societies of Phi Kappa Phi, Tau
Beta Pi, and Sigma Xi. He is listed in the Who's Who in the Frontiers of
Science and Technology, Who's Who of in American Education, American Men and
Women of Science, Who’s Who in the World and other similar publications.
Michael Jordan (University of California,
Berkeley)
Kernel
Independent Component Analysis
Francis R. Bach and Michael I. Jordan (University of California, Berkeley)
We present a new class of algorithms for independent component analysis (ICA) which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space. On the one hand, we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence. On the other hand, building on recent developments in kernel methods, we show that these criteria and their derivatives can be computed efficiently. Minimizing these criteria leads to flexible and robust algorithms for ICA. We illustrate with simulations involving a wide variety of source distributions, showing that our algorithms outperform many of the presently known algorithms.
Michael I. Jordan is Professor in the Department of Electrical Engineering and Computer Science and the Department of Statistics at the University of California at Berkeley. He received his Masters in Mathematics from Arizona State University, and earned his PhD in Cognitive Science from the University of California, San Diego. He was a professor at the Massachusetts Institute of Technology from 1988 to 1998. Prof. Jordan's research has spanned a number of fields, including psychophysics, computer science and statistics. He has published over 140 research articles in these areas. In psychophysics, he is known for his work on the role of predictive forward models in human motor control. In computer science and statistics, he has been a leading figure in the area of probabilistic graphical models, a marriage of graph theory and probability theory that is of wide current interest. Prof. Jordan has given invited plenary lectures at numerous international conferences, including the International Conference on the Mathematical Theory of Networks and Systems, the International Joint Conference on Neural Networks, the ACM Conference on Computational Learning Theory, and the Conference on Uncertainty in Artificial Intelligence.
Gilles Laurent (California Institute of
Technology)
Dynamics
and computation in olfactory circuits
The study of brain function has for a long time ignored the potential importance of neural circuit dynamics. I will examine this issue in the context of olfactory processing. The talk will summarize our data and its interpretation on slow temporal patterning and fast oscillatory synchronization in olfactory circuits. I will show that slow patterning allows input decorrelation (and thus optimization of the use of coding space), while synchronization plays an essential role in the sparsening of representations, a step probably essential for memory formation and pattern matching.
French citizen, born and grown up in Morocco, educated in France: Veterinary medicine + PhD (Neuroethology). Postdoc in Cambridge (Neuroscience, with Malcolm Burrows) for 4 years. At Caltech since 1990, where I am now Professor of Biology and Computation and Neural Systems.
Robert Hecht-Nielsen (HNC & UCSD)
A Real-Time Language-Knowledge-Dependent Solution to the Cocktail Party
Problem
Cortronic architectures consist of
linked classical associative memory structures. Learning is carried out such that
associative recalls optimize significance; a new
information-theoretic quantity. Information processing algorithms (thought
processes), which can be learned by rehearsal practice, are sequences of
regional feature detector and associative fascicle activations analogous to the
muscle contraction sequences of movement (thought is viewed as a phylogenetic
outgrowth of movement). In the cocktail
party problem a cortronic architecture is used at each moment to predict
possible next sound utterances. The feature detectors associated with the
detection of these sounds are sensitized to form an expectation.
If any of the expected sounds arrives the relevant feature detectors respond
and this low-level input is sent on to higher levels for analysis. This analysis
yields an ongoing output of what is being said by the attended-to speaker (and
an ongoing feedback of predictive expectation). There is no source
separation, per se, since other sounds present are not detected in
the first place. The characteristics of this cocktail party
solution match the known characteristics of the human solution at many points;
including a lack of processing delay and a need to guess (or momentarily
increase the SNR) to start the process.
Affiliations:
(1985 - Present) University of California, San Diego
[Teaching Professor]
Program in Computational Neurobiology
Institute for Neural Computation
Department of Electrical and Computer Engineering
(1986 - Present) HNC Software Inc. [Successful Capitalist]
Honors:
Graduate Teaching Award
IEEE Fellow
IEEE Neural Networks Pioneer Award
Education:
Ph.D., Mathematics, Arizona State University
Tony
Bell (The Salk Institute & UCSF)
The Generative Model
I will examine the notion that the goal of the perceptual system is to build a model of how our mixed-up experiences are the result of changes in external objects. There will be examples drawn from mathematics and from neuroscience.
Tony Bell was raised in Northern Ireland and studied computer science and philosophy at the University of St Andrews in Scotland. He received his PhD from the Free University of Brussels for a thesis on "Self-organising Neural Dynamics" in 1993. He did his postdoctoral work on ICA with Terry Sejnowski's Computational Neurobiology lab at the Salk Institute in San Diego. He has also worked in the San Francisco area.
Bhaskar D. Rao (University of California, San Diego)
Algorithms
for Computing Sparse Solutions to Linear Inverse Problems
This paper examines the theoretical and computational issues that arise in signal processing problems with the sparseness constraint in several important application domains. Due to the NP-complete nature of the optimization problem, several suboptimal methods with lower computational complexity have been developed to compute sparse solutions. First, sequential selection methods such as the Matching Pursuit algorithm are discussed. Then, algorithms based on minimizing suitable diversity measures are presented. In particular, algorithms such as FOCUSS (FOCal Underdetermined System Solver) that are based on a factored representation of the gradient and employ an Affine Scaling Transformation are discussed. Finally, some observations are made about the relative performance of the algorithms.
Bhaskar D. Rao received the B. Tech.
degree in electronics and electrical communication
engineering from the Indian Institute of Technology,
Kharagpur, India, in 1979, and the M.S. and Ph.D. degrees from
the University of Southern California in 1981 and 1983
respectively.
Since 1983, he has been with the University of California, San Diego,
where he is currently a Professor in the
Electrical and Computer Engineering department. He is a fellow of
IEEE and has been a member of the Statistical Signal and Array Processing
technical committee. He is currently a member of the
signal processing theory and methods technical
committee. His interests are in the areas of digital signal
processing, estimation theory, and optimization theory, with
applications to digital communications, speech signal
processing, and human-computer interactions.
Christof Koch (California Institute
of Technology)
Responses Of Single Neurons In The Human
Medial Temporal Lobe During Visual
Stimulation, Imagery And Flash Suppression
In collaboration with Dr. Itzhak Fried at UCLA Medical School, we studied the visual properties of single neurons in the medial temporal lobe of patients with intractable epilepsy. Based on clinical criteria, intracranial electrodes were implanted by Dr. Fried to localize seizure foci for surgical resection. Probes were stereotactically placed in bilateral medial temporal lobe targets including hippocampus, amygdala, entorhinal cortex and parahippocampal gyrus. We recorded the activity of individual neurons while patients viewed pictures of faces, objects, patterns and animals on a monitor, when the patients were asked to imagine these pictures with closed eyes and during both rivalry and flash suppression. In this large-scale study of visual responses in more than 800 units in conscious humans, we show that individual neurons respond very selectively to visual stimuli from different natural categories, including faces, We show that a subset of these cells have the identical selectivity during imagery, We also show that the majority of the visually selective cells follows the percept during flash suppression. None of these neurons were active for a perceptually suppressed stimulus.
I was born on November 13, 1956 in Kansas City, Missouri. I subsequently grew up in Amsterdam/Holland, Bonn/Germany, Ottawa/Canada, and Rabat/Marocco where I graduated from the Lycèe Descartes with a French Baccalaurèat (Section C) in 1974.
I studied Physics and Philosophy in Tübingen, Germany, and was awarded my Master of Physics in 1980 and my PhD (Nonlinear information processing in dendritic trees of arbitrary geometry) from the Max-Planck-Institut for Biological Cybernetics in Tübingen in 1982. My "Doctor-Father" was Prof. Tomaso Poggio, now at MIT.
After 4 years as a post-doctoral fellow at the Artificial Intelligence Laboratory and at the Department of Psychology at MIT, I joined Caltech's newly started Computation and Neural Systems option, where I have been ever since (now as a Professor of Computation and Neural Systems), running the K-Lab as well as being the Executive Officer of CNS.
Geoffrey Hinton
(University of Toronto)
We present a new way of interpreting ICA as a probability density model and a new way of fitting this model to data. The advantage of our approach is that it suggests simple, novel extensions to overcomplete, undercomplete and multilayer non-linear versions of ICA.
Geoffrey Hinton received his BA in experimental psychology from Cambridge in 1970 and his PhD in Artificial Intelligence from Edinburgh in 1978. He did postdoctoral work at Sussex University and the University of California, San Diego and spent five years as a faculty member in the Computer Science department at Carnegie-Mellon University.
He then moved to Toronto where he was a fellow of the Canadian Institute for Advanced Research and a Professor in the Computer Science and Psychology departments. He is a former president of the Cognitive Science Society and a fellow of the Royal Society of Canada and of the American Association for Artificial Intelligence. In 1992 he won the ITAC/NSERC award for contributions to information technology. A simple introduction to his research can be found in his articles in Scientific American in September 1992 and October 1993.
He does research on ways of using neural networks for learning, memory, perception and symbol processing and has over 100 publications in these areas. He was one of the researchers who introduced the back-propagation algorithm that is now widely used for practical applications. His other contributions to neural network research include Boltzmann machines, distributed representations, time-delay neural nets, mixtures of experts, and Helmholtz machines. His current main interest is in unsupervised learning procedures for neural networks with rich sensory input
ABSTRACT
Jean-Francois Cardoso
The Three Easy
Routes to Independent Component Analysis; Contrasts and Geometry
Blind separation of independent sources can be achieved by exploiting non Gaussianity, non stationarity or time correlation. This paper examines in a unified framework the
objective functions associated to these three routes to source separation. They are the ‘easy routes’ in the sense that the underlying models are the simplest models able to capture the statistical structures which make source separation possible. A key result is a generic connection between mutual in-formation, correlation and marginal ‘non properties’: non Gaussianity, non stationarity, non whiteness.
Deniz Erdogmus, José C. Príncipe
An On-Line Adaptation Algorithm for Adaptive System Training with Minimum Error Entropy: Stochastic Information Gradient
We have recently reported on the use of minimum error entropy criterion as an alternative to minimum square error (MSE) in supervised adaptive system training. A nonparametric estimator for Renyi’s entropy was formulated by employing Parzen windowing. This
formulation revealed interesting insights about the process of information theoretical learning, namely information potential and information forces. Variants of this criterion
were applied to the training of linear and nonlinear adaptive topologies in blind source separation, channel equalization, and chaotic time-series prediction with superior results. In this paper, we propose an on-line version of the error entropy minimization algorithm, which can be used to train linear or nonlinear topologies in a supervised fashion. The algorithms used for blind source separation and deconvolution can be modified in a similar fashion. For the sake of simplicity, we present preliminary experimental results for FIR filter adaptation using this on-line algorithm and compare the performance with LMS.
Riccardo Boscolo, Hong Pan, Vwani P. Roychowdhury
We introduce a novel approach to the blind signal separation (BSS) problem that is capable of jointly estimating the probability density function (pdf) of the source signals
and the unmixing matrix. We demonstrate that, using a kernel density estimation based Projection Pursuit (PP) algorithm, it is possible to extract, from instantaneous mixtures,
independent sources that are arbitrarily distributed. The proposed algorithm is non-parametric, and unlike conventional Independent Component Analysis (ICA) frame-works, it requires neither the definition of a contrast function, nor the minimization of the high-order cross-cumulants of the reconstructed signals. We derive a new method for solving the resulting constrained optimization problem that is capable of accurately and efficiently estimating the unmixing matrix, and which does not require the selection
of any tuning parameters. Our simulations demonstrate that the proposed method can accurately separate sources with arbitrary marginal pdfs with significant performance gain when compared to existing ICA algorithms. In particular, we are successful in separating mixtures of skewed, almost zero-kurtotic signals, which other ICA algorithms fail to separate.
Noboru Murata
Properties of the Empirical Characteristic Function and its Application to Testing for Independence
In this article, the asymptotic properties of the empirical characteristic function are discussed. The residual of the joint and marginal empirical characteristic functions is studied and the uniform convergence of the residual in the wider sense and the weak convergence of the scaled residual to a Gaussian process are investigated. Taking into account of the result, a statistical test for independence against alternatives is considered.
Vincente Zarzoso, Asoke K. Nandi
A General Theory of Closed-Form Estimators for Blind Source Separation
We present a general theory for the closed-form parametric estimation of the unitary mixing matrix after pre-whitening in the blind separation of two source signals from two noiseless instantaneous linear mixtures. The proposed methodology is based on the algebraic formalism of bicomplex numbers and is able to treat both real and complex valued mixtures indiscriminately. Existing analytic methods are found as particular cases of the exposed unifying formulation. Simulations in a variety of separation scenarios— even beyond the noiseless two-signal case — compare, assess and validate the methods studied.
Yasu Matsuyama, Naoto Katsumata, Shuichiro Imahara
Convex Divergence as a Surrogate Function for Independence: The f-Divergence ICA
The convex divergence is used as a
surrogate function for obtaining independence of random variables described by
the joint probability density. If the kernel convex function is twice
continuously differentiable, this case reveals a class of generalized logarithm.
This class of logarithms gives generalizations of the score function and the
Fisher information matrix which are related to the Cramer-Rao bound. Guided by
these properties, independent component analysis (ICA) using the convex
divergence is presented.Obtained algorithms use the past and/or future
data. Software implementation is easy
and beats the minimum mutual information ICA in the speed. Real world
experiments on brain fMRI are also performed.
Dinh Tuan Pham
Contrast Functions for Blind Separation and Deconvolution of Sources
A general method to construct contrast functions for blind source separation is presented. It is based on a super-additive functional of class II applied to the distributions of the reconstructed sources. Examples of such functionals are given. Our approach permits exploiting the temporal dependence of the sources by using a functional on the joint distribution of the source process over a time interval. This yields many new examples and frees us from the constraint that the sources be non Gaussian. Contrasts functions based on cumulants requiring the orthogonality constraint is also covered. Finally, the case of convolutive mixtures is considered in relation with the problem of blind separation-deconvolution.
Maria Funaro, Erkki Oja, Harri Valpola
Artefact Detection in Astrophysical Image Data Using Independent Component Analysis
This paper is the first reported application of ICA on astro-physical image data. When studying far-out galaxies from a series of consequent telescope images, there are several sources for artefacts that influence all the images such as camera noise, atmospheric fluctuations and disturbances, and stars in our own galaxy. For this problem, the linear ICA model holds very accurately, because the independence of such artefacts is guaranteed. Using image data on the M31 Galaxy, it is shown that several clear artefacts can be de-tected and recognized based on their temporal pixel luminosity profiles and independent component images. Once these are removed, it is possible to concentrate on the real physical events like gravitational lensing. ICA might pro-vide a very useful preprocessing for the large amounts of available telescope image data.
Milutin Stanacevic, Gert Cauwenberghs, George Zweig
Gradient Flow Broadband Beamforming and Source Separation
We present and demonstrate a method for blind separation and bearing estimation of broadband traveling waves, impinging on a sensor array with dimensions smaller than the shortest wavelength in the sources. By sensing spatial and temporal gradients of the received signal, the problem of separating mixtures of time-delayed sources reduces to that of separating instantaneous mixtures of the gradient components
of the sources using conventional tools of independent component analysis. Experimental results demonstrate real-world separation of speech in outdoors and indoors environments, using a planar array of four microphones within a 5 mm radius and analog circuits computing spatial and temporal derivatives.
Tapani Ristaniemi, Rui Wu
Mitigation of
Continuous-Wave Jamming in DS-CDMA Systems Using Blind Source Separation
Techniques
In this paper we consider blind interference suppression indirect sequence spread spectrum communication system with co-existing continuous wave (CW) jammer. Especially, blind source separation techniques are utilized. However, unlike in recent works only a single antenna element is employed. Numerical examples are given to illustrate achieved performance gains.
Xiaoan Sun, Scott C. Douglas
A Natural Gradient Convolutive Blind Source Separation Algorithm for Speech Mixtures
In this paper, a novel algorithm for separating mixtures of multiple speech signals measured by multiple microphones in a room environment is proposed. The algorithm is a modification of an existing approach for density-based multichannel blind deconvolution using natural gradient adaptation. It employs linear predictors within the coefficient updates and produces separated speech signals whose autocorrelation properties can be arbitrarily specified. Stationary point analyses of the proposed method illustrate that, unlike multichannel blind deconvolution methods, the proposed algorithm maintains the spectral content of the original speech signals in the extracted outputs. Performance comparisons of the proposed method with existing techniques show its desirable properties in separating real-world speech mixtures.
Wolf Baumann, Bert-Uwe Kohler, Dorothea Kolossa and Reinhold Orglmeister
Real Time Separation of Convolutive Mixtures
A new concept of combining conventional beamforming with independent component analysis (ICA) techniques and its implementation on a multi DSP system is presented. The system consists of two floating point digital signal processors TMS320C6701, an eight channel linear microphone array, an analog/digital converter board and a handheld control unit for stand alone operation. In the two system stages a sum and delay beamformer as well as a convolutive ICA algorithm are implemented. Due to the high performance of the digital signal processors, the systems achieves blind separation of two convolutive mixed sources in real time.
Ki-Seok Cho, Soo-Young Lee
Implementation
of INFOMAX ICA Algorithm with Analog CMOS Circuits
Independent Component Analysis algorithm based on infomax theory with natural gradient was implemented with a fully-analog CMOS chip. Although one chip consists of 4 inputs and 4 outputs, the chip incorporates fully-modular architecture for multi-chip applications. The fabricated chip demonstrated improved SNRs for unknown speech mixtures.
Frank Meinecke, Andreas Zieche, Motoaki Kawanabe, Klaus R.Mueller
Assessing Reliability of ICA Projections – A Resampling Approach
When applying unsupervised learning techniques like ICA or temporal decorrelation for BSS, a key question is whether the discovered projections are reliable. In other words: can we give error bars or can we assess the quality of our separation? We use resampling methods to tackle these questions and show experimentally that our proposed variance estimations are strongly correlated to the separation error. We demonstrate that this reliability estimation can be used to choose an appropriate ICA-model, to enhance significantly the separation performance, and, most important, to mark the components that can really have a physical meaning. An application to data from an MEG 1 -experiment underlines the usefulness of our approach.
Harold Szu
A Priori Maxent H(S) Independent Class Analysis (ICA) Vs. A Posteriori Maxent (V) ICA
Two mirror symmetric versions of the maximum entropy (MaxEnt) methodology are introduced and compared: (1) A posteriori MaxEnt Independent Component Analysis (ICA) H(V) was proposed by Bell, Sejnowski, Amari, Oja (BSAO) (early by Jutten & Herault, Comon and Cardoso (JHCC) in France). It is ambitious to factorize the unknown joint-probability density function (j-pdf) using the post processing algorithm involving the sigmoid-threshold neurons' output V(x,y)= s([W]X(x,y)) of all image locations (x,y) in order to apply the pixel-ensemble averaged synaptic weight matrix [W] learning, ¶[W]/¶t=<¶H(V)/¶[W]>. The pixel ensemble average may be necessary to factorize the unknown joint-pdf from multi-channel data vector X(x,y). (2) A priori MaxEnt H(S) for independent class analysis (ica) is a compliment first step to the ambitious joint-pdf factorization based on a-posteriori MaxEnt H(V) ICA. Since ica is less ambitious to ICA in finding independent classes alone without their underlying pdf, we can derive from Gibb's statistics mechanics of independent classes of irradiation sources Sj by a priori MaxEnt H(S), which would be a flat equal class distribution if each were not constrained by the measurements by means of Lagrangian multipliers of force vector li and energy scalar (lo -1) for each pixel: ( Entropy Equation: See Paper).
Physically speaking, the long distance propagation by the speed of the light insures the line of sight local validity of a linear and instantaneous within-pixel mixture model of remote optical sensing. However, only the a priori MaxEnt H(S) ica can handle a large hyperspectral image data basis by the divide-and-conquer strategy without the pixel ensemble average that limits computationally the posteriori MaxEnt ICA algorithm. This strategy is possible because the radiation sources vector S(xo,yo) per pixel contribute locally to the spectral image data per pixel X(xo,yo)= [A]S(xo,yo), i.e. the irradiation collected within the foot print of the individual pixel (xo,yo) will not mix with other neighborhood irradiation sources that will only contribute to their corresponding neighborhood pixels. Although this is basically true for any optical imaging, the post processing BSAO algorithm attempts to de-mix all pixel data [W]X in a batch mode that make it not scalable to large image data basis. Being a priori MaxEnt H(S) pre-processing, we can divide and conquer image size by applying ica pixel by pixel in real time. Furthermore, we have derived ab initio from it the usual sigmoid threshold logic S = s(l[A]) and the Hebbian learning ruled DAij =liSj. We can thus conjecture any linear communication theorem that a linear matrix transform (e.g. associative memory [A] recall) between data X=[A]S and its independent classes under the constraint of comprehensive decomposition Sj Sj = 1 must lead naturally to the coupling of sigmoid transfer logic and Hebbian learning. Thus the author has coined the Lagrangian Constraint Neural Network (LCNN) since 1997. Nonlinear ICA generalization by LCNN and two conjugate gradient ascents of two mirror symmetric MaxEnt's are indicated.
R.A. Choudrey and S.J. Roberts
Flexible Bayesian Independent Component Analysis for Blind Source Separation
Independent Component Analysis (ICA) is an important tool for extracting structure from data. ICA is traditionally performed under a maximum likelihood scheme in a latent variable model and in the absence of noise. Although extensively utilised, maximum likelihood estimation has well known drawbacks such as overfitting and sensitivity to local-maxima. In this paper, we propose a Bayesian learning scheme using the variational paradigm to learn the parameters of the model, estimate the source densities, and –together with Automatic Relevance Determination (ARD) - to infer the number of latent dimensions. We illustrate our method by separating a noisy mixture of images, estimating the noise and correctly inferring the true number of sources.
Albert Bijaoui, Danielle Nuzillard
Smoothing and
Adaptive Denoising for Blind Source Separation
Pixel unmixing of multispectral astronomical images was examined as a blind source separation of an instantaneous mixture. The capabilities of separation algorithms were tested on different simulatd images. The results showed that only in case of a high signal-to noise ratio fine separations can be carried out. In this communication, the improvements resulting from a pre denoising are examined. Optimal linear smoothing and two adaptive wavelet denoisings were applied before separation. The increase of the separation quality of the obtained sources is discussed according to both the separation algorithms and the denoising methods.
Stefan Harmeling, Andreas Ziehe, Motoaki Kawanabe, Benjamin Blankertz, Klaus
Robert Muller
Nonlinear Blind
Source Separation Using Kernel Feature Spaces
In this work we propose a kernel-based blind source separation (BSS) algorithm that can perform nonlinear BSS for general invertible nonlinearities. For our kTDSEP algorithm we have to go through four steps: (i) adapting to the intrinsic dimension of the data mapped to feature space F , (ii) finding an orthonormal basis of this submanifold, (iii) mapping the data into the subspace of F spanned by this orthonormal basis, and (iv) applying temporal decorrelation BSS (TDSEP) to the mapped data. After demixing
we get a number of irrelevant components and the original sources. To find out which ones are the components of interest, we propose a criterion that allows to identify the
original sources. The excellent performance of kTDSEP is demonstrated in experiments on nonlinearly mixed speech data.
J. Eriksson, A. Kankainen, V. Koivunen
Novel Characteristic Function Based Criteria for ICA
We introduce two nonparametric independent component analysis (ICA) criteria based on factorization of characteristic functions. This approach has potential to separate wide
class of distributions because characteristic function always exists. A simple criterion allowing for efficient search of the separating matrix and a more advanced criterion possessing desirable consistency property are presented. These criteria may easily be used in orthogonal ICA algorithms. Separating matrix is estimated by establishing pairwise independence among the source signals. Theoretical characteristic functions used in the criteria are replaced by empirical ones. In the examples, the reliable performance of the methods is demonstrated using a variety of source distributions including skewed and heavy-tailed distributions.
Toshinao Akuzawa
New Fast Factorization Method for Multivariate Optimization and its Realization as ICA Algorithm
A new
framework for multivariate optimization by crite-ria invariant under
componentwise scaling is constructed. These problems are naturally considered
as problems on the coset R^{xN} \ GL (N,R). We show that there is a duality
between the optimization flow on this coset and the dynamics of quantum lattice
with inner degrees of freedom. Then we propose a new algorithm for optimization
problems on this coset named nested Newton’s method, whose essence is the
ignorance of n-body interactions with n≥3
if we explain it by using its analogy to quantum lattice. This method is
readily applicable as a highly useful ICA algorithm, which is robust under
gaussian noises, quite fast, and practical for large dimensional problems. The
last feature comes from
the fact that our method requires memory space of order N2, whereas the conventional Newton’s method for ICA (and the JADE) requires that of order N4. The matlab code
for this algorithm is available from our web-site.
Wei Lu, Jagath C. Rajapakse
We present a novel approach to extract a subset of independent sources from multidimensional observations when some a priori information that can be incorporated to the learning algorithm as reference is available. The constrained independent component analysis (cICA) is extended to use new constraints, and a Newton-like learning algorithm is proposed to give an optimal solution to the constrained optimization problem. The convergence and the effect of parameters of the learning algorithm are analyzed. Simulations with the mixtures of deterministic and random signals and synthetic fMRI data demonstrate the efficacy and accuracy of the proposed algorithm.
Kenneth E. Hild II, Deniz Erdogmus, Jose C. Principe
On-Line Minimum Mutual Information Method for Time-Varying Blind Source Separation
The MeRMaId (Minimum Renyi’s Mutual Information) algorithm for BSS (blind source separation) has previously been shown to outperform several popular algorithms in
terms of data efficiency. The algorithms it compared favor-ably with include Hyvaerinen’s FastICA, Bell and Sejnowski’s Infomax, and Comon’s MMI (Minimum Mutual Information) methods. The drawback is that the MeRMaId algorithm has a computational complexity of O(L 2 ), as compared to O(L) for the other three. However, a new advancement referred to as SIG (Stochastic Information Gradient), can be used to modify the MeRMaId criterion such that the complexity is reduced to O(L). The
modified criterion is then applied to the separation of instantaneously mixed sources using an on-line implementation. Simulations demonstrate that the new algorithm preserves the separation performance of the original algorithm and, in fact, compares quite favorably with several published methods.
Shoko Araki, Shoji Makino ,Ryo Mukai, Tsuyoki Nishikawa, Hiroshi Saruwatari
Fundamental Limitation of Frequency Domain Blind Source
Separation for Convolved Mixture of Speech
Despite several recent proposals to
achieve Blind Source Separation (BSS) for realistic acoustic signals, the
separation performance is still not enough. In particular, when the length of
an impulse response is long, the performance is highly limited. In this paper,
we consider the reason for the p r performance of BSS in a long reverberation
environment. First, we show that it is useless to be constrained by the
condition P T, where T is the frame size f FFT and P is the length of a r m
impulse response. We also discuss the limitation of frequency domain BSS, by
showing that the frequency domain BSS framework is equivalent to two sets of
frequency domain adaptive beamformers.
Massoud Babaie-Zadeh, Christian Jutten, Kambiz Nayebi
Blind Separating
Convolutive Post Non-Linear Mixtures
This paper addresses blind source separation in convolutive post nonlinear (CPNL) mixtures. In these mixtures, the sources are mixed convolutively, and then measured by
nonlinear (e.g. saturated) sensors. The algorithm is based on minimizing the mutual information by using multivariate score functions.
Radu Balan, Justinian Rosca, Scott Rickard
Robustness of Parametric Source Demixing in Echoic Environments
Blind source separation (BSS) of audio signals in echoic environments such as an office room is still a very challenging problem. Here we approach the problem from a practical perspective and shed light on how robust a two channel echoic parametric demixing can get. We assume that an oracle (i.e. a perfect estimator) provides a truncated estimate of
the mixing FIR filters for a given source configuration. This way we can study the properties of a parametric demixer using the adjoint of the truncated mixing matrix. For several degrees of truncation, we compute how the separation SNR varies as a function of the uncertainty of the true source position. The true source position is uniformly distributed within a sphere of radius R around an assumed position, to reflect the fact that parameters of interest are imprecisely estimated. Simulations of artificial echoic mixings show that the higher order demixing filters have little robustness to position uncertainties (and therefore to errors of estimation) while the overall performance remains almost constant beyond the second order approximation. This should represent a guideline for what is practically achievable with a class of BSS techniques in echoic environments.
Allan Kardec Barros, Noboru Ohnishi
Fetal Heart Rate
Variability Extraction by Frequency Tracking
In this work, we propose an algorithm to extract the fetal heart rate variability from an ECG measured from the mother abdomen. The algorithm consists of two methods: a separation algorithm based on second-order statistics that extracts the desired signal in one shot through the data, and a hearth instantaneous frequency (HIF) estimator. The HIF algorithm is used to extract the mother heart rate which serves as reference to extract the fetal heart rate. We carried out simulations where the signals overlap in frequency and time, and showed that the it worked efficiently.
V. Calhoun, T. Adali, G. Pearlson, J. Pekar
Group ICA of
Functional MRI Data: Separability, Stationarity, and Inference
Independent component analysis (ICA) is being increasingly applied to functional MRI (fMRI) data. A principal advantage of this approach is its applicability to cognitive paradigms for which detailed a priori models of brain activity are not available. ICA has been successfully utilized to analyze single-subject fMRI data sets, and we have recently extended this work to provide for group inferences. In order to perform group analysis, we concatenate the single-subject images in time and perform a single ICA estimation, then back-reconstruct individual subject maps and time courses. When applied to fMRI data acquired during a simple visual paradigm, our group ICA analysis revealed task-related components in left and right visual cortex, a transiently task-related component in bilateral occipital/parietal cortex, and a non task-related component in bilateral visual association cortex. In this work, we develop three important areas needed for applying ICA to group data: separability, stationarity, and inference. Our results further demonstrate the utility of using such a method for making group inferences on fMRI data using ICA.
Lai-Wan Chan, Siu-Ming Cha
Selection of
Independent Factor Model in Finance
In finance, factor model is a fundamental model to describe the return generation process. Traditionally, the factors are assumed to be uncorrelated with each other. We argue that independence is a better assumption to factor model from the viewpoint of portfolio management. Based on this assumption, we propose the independent factor model. As the
factors are independent, construction of the model would be another application of Independent Component Analysis (ICA) in finance. In this paper, we illustrate how we
select the factors in the independent factor models. Securities in the Hong Kong market were used in the experiment. Minimum description length (MDL) was used to select the number of factors. We examine four sorting criteria for factor selection. The resultant models were cross-examined by the runs test.
Changkyu Choi
Speech Enhancement
Using Sparse Code Shrinkage and Global Soft Decision
This paper relates to a method of enhancing speech quality by eliminating noise in speech presence intervals as well as in speech absence intervals based on speech absence prob-ability. To determine the speech presence and absence intervals, we utilize the global soft decision. This decision makes the estimated statistical parameters of signal density models more reliable. Based on these parameters the noise suppressor equipped with sparse code shrinkage functions reduces noise considerably in real-time.
Andrzej Cichocki, Adel Belouchrani
Source Separation of Temporally Correlated Source Using Bank of Band Pass Filters
This paper introduces a new source separation algorithms exploiting the difference in the spectra shapes of the source signals. The proposed approach relies only on second-order statistics and estimates the mixing matrix by using eigen-value decomposition of covariance matrix of prewithened sensor signals or alternatively an input output identification procedure using as inputs linear band pass filtered versions of the estimated colored sources. An adaptive implementation of the proposed technique is presented. The new algorithm shows to be computationally very simple and efficient. In addition and in contrast to other existing techniques, the covariance of the noise do not need to be modeled. The effectiveness of the proposed method is illustrated by some numerical simulations.
Adriana Dapena, Monica F. Bugallo, Luis Castedo
Separation of Convolutive Mixtures of Temporally-White Signals: A Novel Frequency-Domain Approach
In this paper we propose a novel approach for separating convolutive mixtures in the frequency domain. This approach involves the solution of several instantaneous mixing
problems and the elimination of the indeterminacies which appear because the sources may be extracted in a different order or with different amplitudes in some frequency bins.
In order to separate each instantaneous mixture, we will extend the criterion proposed in [4]. We also show that both the permutation and the amplitude indeterminacies can be removed using second-order statistics when the sources are temporally-white.
Akira Date
An Information Theoretic Analysis of 256-Channel EEG Recordings: Mutual-Information and Measurement Selection Problem
How does one part affect another in the brain? How much information can we extract from the brain data? The multi-channel EEG recording system is now available to study this issue. We recorded 256-channel EEGs while a subject performed a visual discrimination task, and obtained mutual in-formation between a visual stimulus condition and signals from single and multiple EEGs. Here we report preliminary results which show a power of the 256 channel recording system. In particular, we show examples that the best two informative independent measurements are not the two best.
Deniz Erdogmus, Luis Vielva, Jose Principe
Nonparametric
Estimation and Tracking of the Mixing Matrix for Underdetermined Blind Source
Separation
Blind source separation deals with the problem of estimating n source signals from m measurements, which are generated through an unknown mixing process. In the underdetermined linear case, where the number of measurements is smaller than the number of sources, the solution can be obtained in three stages: find a sparse representation domain for the signals, find the mixing matrix, and estimate the sources using the previous knowledge. This paper addresses the second stage. A non-parametric maximum-likelihood approach based on Parzen windowing is presented. It is shown that the peak points in the probability distribution of measurements directions correspond to the directions of the column vectors of the mixing matrix. An algorithm to estimate the column vectors in the static case, and to track the column vectors in the dynamic case is presented. The tracking capability of the algorithm is determined and, using a simple wave propagation model, corresponding limitations on the speeds of mobile sources are derived.
Simone Fiori
Some Properties of
Bell-Sejnowski PDF-Matching Neuron
The aim of the present paper is to investigate the behavior of a single-input single-unit system, learning through the maximum-entropy principle, in order to understand some
formal property of Bell-Sejnowski’s PDF-matching neuron. The general learning equations are presented and two case-study are discussed with details.
V. V. Gavrik
Criteria for a Linear Structure of Variance and Obtaining the Process Functions from Principal Components
The description of multivariate data with principal components allows most accurately to approximate a random function with a given number of expansion terms. The question is how to establish that the biorthogonal expansion is really finite in the physical sense and an experimental function is a random linear combination of several non-random functions over the instrumental error. Many situations are expected to result in the linear variance structure or be reducible to it on the base of physical considerations. Statistical criteria have been elaborated to detect the situations since a data variance could not be directly seen to possess the linear structure even in two-dimensional situations. Some analytical properties of principal components are shown to give reliable rotation criteria for calculation the informal process functions. The approach is illustrated with the variance-linearization and rotation transforms for large sets of experimental dependencies describing the leap-like changes in the yield of a chemical process at the atom-by-atom growth of catalyst nanoclusters. Facing the latest results on the neural algorithms of identification and decision-making, the approach appears to promise efficiently to decode the large arrays of data on the electrical activities of brain.
Michel Haritopoulos, Hujun Yin, Nigel M. Allinson
Multiplicative Noise Removal
Using Self-Organizing Maps
This paper approaches the problem of image denoising from an Independent Component Analysis (ICA) perspective. Considering that the pixels intensity constituting the crude images represents the useful signal corrupted with noise, we show that, a nonlinear ICA-based approach can provide a satisfactory solution to the Non-Linear Blind Source Separation problem (NLBSS). Self-Organizing Maps (SOMs) are well suited for performing this task, due to their nonlinear mapping property. Separation results obtained from test images demonstrate the feasibility of the proposed method.
Robert Jenssen, Tor Arne Oigš ard, Torbjorn Eltoft and Alfred Hanssen
Sparse Code Shrinkage Based on the Normal Inverse Gaussian Density Model
In this paper we introduce the recent normal inverse Gaussian (NIG) probability density as a new model for sparsely coded data. The NIG density is a flexible, four-parameter
density, which is highly suitable for modeling unimodal super-Gaussian data. We demonstrate that the NIG density provides a very good fit to the sparsely coded data, obtained here via an independent component analysis (ICA) transform of the observations. In image denoising, we utilize this new density by developing a NIG-based maximum a posteriori estimator of a sparsely coded image corrupted by white Gaussian noise. The estimator acts as a shrinkage operator on the noisy components in the sparse domain. We demonstrate the technique by an image denoising experiment.
Julia Karvanen, Visa Koivunen
Blind Separation
Using Absolute Moments Based Adaptive Estimating Function
We propose new absolute moment based estimating functions for blind source separation purposes. Absolute moments are a computationally simple choice that can also adapt to the skewness of source distributions. They have lower sample variance than cumulants employed in many widely used ICA (Independent Component Analysis) methods. The complete estimating function consists of two parts that are sensitive to peakedness and asymmetry of the distribution, respectively. Expression for optimal weighting between the parts is derived using an efficacy measure. The performance of the proposed contrast and employed efficacy measure are studied in simulations.
Peter Meinicke, Helge Ritter
Independent
Component Analysis with Quantizing Density Estimators
We propose an approach to source adaptivity in ICA based on quantizing density estimators (QDE). These estimators allow torealize source adaptivity in an efficient and non-parametric way and we show how their application can be viewed as a natural extension to recent approaches based on parametric models. In simulations we show that ICA based on QDE can considerably increase the performance of blind source separation as compared with flexible parametric approaches.
Ryo Mukai, Shoko Araki, Shoji Makino
Separation and Dereverberation Performance of Frequency Domain Blind Source Separation
In this paper, we investigate the separation and dereverberation performance of frequency domain Blind Source Separation (BSS) based on Independent Component Analysis (ICA) by measuring impulse responses of a system. Since ICA is a statistical method, i.e., it only attempts to make outputs independent, it is not easy to predict what is going on in a BSS system physically. We therefore investigate the detailed components in the processed signals of a whole BSS system from a physical and acoustical viewpoint. In particular, we focus on the direct sound and reverberation in the target and jammer signals. As a result, we reveal that the direct sound of a jammer can be removed and the reverberation of the jammer can be reduced to some degree by BSS, while the reverberation of the target cannot be reduced. Moreover, we show that a long frame length causes pre-echo noise, and this damages the quality of the separated signal.
0
Satoshi Nakashima, Kiyotoshi Matsuoka
An Efficient
Algorithm for Adaptive Separation of Mixture of Speech Signals
This
paper proposes an efficient algorithm for blind source separation (BSS) of
mixture of speech signals. Conventional on-line algorithms for blind separation
usually assume that the sources are iid or linear processes. Since however
speech signals have strong nonlinearity, those algorithms are not efficient
with respect to convergence and sometimes induce instability. In order to solve
these issues we introduce a more suitable probabilistic model for speech
signals, namely, a speech signal is modeled as an amplitude modulation of a
stationary random process. Based on the model, a new BSS algorithm is derived.
A couple of examples reveal that the proposed algorithm determines a desired
separator within a considerably short time.
Kamran Rahbar, James P. Reilly
Blind Source Separation Algorithm for MIMO Convolutive
Mixtures
We consider the problem of blind source
separation of MIMO convolutive mixtures for the general case where the number
of sensors are greater than or equal to the number of sources. We assume that
sources are non-stationary signals. The separation is performed in the
frequency domain by joint minimization of the off-diagonal elements of observed
signal's cross-spectral density matrices over different epochs. We propose an
efficient Newton-based algorithm over the complex Stiefel manifold to minimize
an appropriate cost function. We resolve the permutation problem using a novel
tree structured diadic detection scheme. We find and correct wrong permutations
at each frequency bin based on cross frequency correlation between diagonal
elements of the output cross spectral matrices. We demonstrate the performance
of the new algorithm using synthetic mixtures and real word recordings. The
method has the additional advantage of fast convergence.
Fathi
M. Salam, Khurram Waheed
State Space Feedforward and Feedback Structures for Blind Source Recovery
This paper presents two separate structures
for the blind source recovery (BSR) of stochastically independent signal
sources. We hypothesize linear state space models for both the mixing
environment and the demixing (i.e. recovering) adaptive network. Separate
algorithms for adaptive estimation of parameters for the feedforward and
feedback recovering networks have been derived. Auxillay conditions for the
convergence of these algorithms have also been derived and discussed.
Simulation examples have been included to compare the results for both
algorithms for an IIR mixing environment. Conclusive remarks about
effectiveness of these techniques in various practical problems have also been
included.
Mirko Solazzi, Raffaele
Parisi, Aurelio Uncini
Blind Source Separation in Nonlinear Mixtures by
Adaptive Spline Neural Networks
In this paper a novel paradigm for blind source separation in the presence of nonlinear mixtures is presented and described. The proposed approach employs a neural model based on adaptive B-spline functions. Signal separation is achieved through an information maximization criterion. Experimental results and comparison with existing solutions confirm the effectiveness of the proposed architecture.
Milutin Stanacevic, Marc Cohen, Gert Cauwenberghs
Blind Separation of Linear Convolutive Mixtures Using Orthogonal Filter Banks
We propose an algorithm and architecture for real-time blind source separation of linear convolutive mixtures using orthogonal filter banks. The adaptive algorithm derives from stochastic gradient descent optimization of a performance metric that quantifies independence not only across the re-constructed sources, but also across time within each source. The special case of a Laguerre section offers a compact representation with a small number of filter taps even under severe reverberant conditions, facilitating real-time implementation in a modular and scalable parallel architecture. Simulations of the proposed architecture and update rule validate the approach.
Kunio Takaya, Kyung-Yung Choi
Detection of Facial
Components in a Video Sequence by Independent Component Analysis
This paper presents two approaches for detecting facial components in the images contained in a video sequence by Independent Component Analysis (ICA). The ultimate objective is to map detected facial components such as eyes and mouth to a 3D wireframe model to be used in facial animation. One approach is to use face localization technique whereas the other deals with an entire image without cropping face part, but ßltering with wavelet subband ßlters. Face localization is achieved by using the human skin color ßltering in YCbCr subspace with other relevant facial information. The wavelet subband ßlter in the latter approach removes facial parts occupying a large area that contribute in ICA as large basis vectors. The method of Independent Component Analysis is then applied to identify main facial components for subsequent geometrical shape analysis.
Erik Visser, Te-Won Lee, Manabu Otsuka
Speech Enhancement
in a Noisy Car Environment
We present a new speech enhancement method for robust speech recognition in a noisy car environment. The method is based on the combination of two important building blocks, namely blind source separation given two microphone signals and speech denoising using a hybrid Wavelet - Independent Component Analysis (ICA) filterbank. The first block separates point sources such as the passenger’s voice signal whereas the second block eliminates distributed noise signals such as road and wind noise. We performed experiments with real recordings taken while driving in a noisy automobile environment. The method works nearly real-time and achieves good separation results.
Patrick M. Wong, Seungjin Choi, Yanping Niu
A Comparison of
PCA/ICA for Data Preprocessing in a Geoscience Application
This paper presents a performance comparison of a variety of data preprocessing algorithms in a geoscience application. The selected algorithms are principal component analysis (PCA) and three different independent component analyses (FLEXICA, JADE and SOBI). These algorithms are applied to a set of electrical and radioactive signals obtained from a drilled well in Indonesia. Standard backpropagation neural networks are used to perform pattern (flow unit) classification from raw or preprocessed data. The results show that use of the preprocessed data gives more confident results than those
obtained from the raw data. Among the preprocessing algorithms, FLEXICA seems to slightly outperform the others. The study also present a technological framework for combining results from different techniques and it shows that further improvement was achieved.
Liqing Zhang, Andrzej Cichocki
Sparse Representation and Blind Deconvolution of Dynamical Systems
In this paper, we discuss blind deconvolution of dynamical systems, described by the state space model. First we formulate blind deconvolution problem in the framework of the state space model. The blind deconvolution is fulfilled in two stages: internal representation and signal separation. We employ two different learning strategies for training the parameters in the two stages. A sparse representation approach is presented based on the independent decomposition. Some properties of the sparse representation approach are discussed. The natural gradient algorithm is used to train the external parameters in the stage of signal separation. The two-stage approach provides a new insight into blind deconvolution in the state-space framework. Finally a computer simulation is given to show the validity and effectiveness of the state-space approach.
Michael Zibulevsky, Yehoshua Y. Zeevi
Source Extraction Using Sparse Representation
It was discovered recently that sparse decomposition by signal dictionaries results in dramatic improvement the qualities of blind source separation. We exploit sparse decomposition of a single souce in order to extract it from multidimensional sensor data, in applications where a rough template of the source is known. This leads to a convex optimization problem, which is solved by a Newton-type method. Complete and overcomplete dictionaries are considered. Simulations with synthetic evoked responses mixed into natural 122-channel MEG data show significant improvement in accuracy of signal restoration.
Sophie Achard, Dinh Tuan Pham, Christian Jutten
Blind Source Separation in Post Nonlinear Mixtures
This work implements alternative algorithms to that of Taleb and Jutten for blind source separation in post nonlinear mixtures. We use the same mutual information criterion as them, but we exploit its invariance with respect to translation to express its relative gradient in terms of the derivatives of the nonlinear transformations. Then we develop al-gorithms based on these derivatives. In a semi-parametric approach, the latter are parametrized by piecewise constant functions. All algorithms require only the estimation of the score functions of the reconstructed sources. A new method for score function estimation is presented for this purpose.
Frédéric Berthommier, Seungjin Choi
Evaluation of CASA and BSS Models for Subband Cocktail-Party Speech Separation
For speech segregation, a blind separation model (BSS) is tested together with a CASA model which is based on the localisation cue and the evaluation of the time delay of arrival (TDOA). The test database is composed of 332 binary mixture sentences recorded in stereo with a static set-up. These are truncated at 1 second for the simulations. For applying the two models, we cut the frequency domain in a variable number of subbands, which are processed independently. Then, we evaluate the gain, using reference signals recorded in isolation. Without using this reference, a coherence index is also established for the BSS model, which measures the degree of convergence. After a careful analysis, we find gains of about 1-3dB for the two methods, which are smaller than those published for the same task. The variation of the number of subbands allows an optimisation, and we obtain a significant peak at 4 subbands for the CASA model, and a smaller maximum at 2 subbands for the BSS model.
J.D. Carew, V.M. Haughton, C.H. Moritz, B.P. Rogers, E.V. Nordheim, M.E. Meyerand
Frequency Domain
Hybrid Independent Component Analysis of Functional Magnetic Resonance Imaging
Data
Independent component analysis (ICA) of functional magnetic resonance imaging (fMRI) data reveals spatially independent patterns of functional activation. The purely data-driven approach of ICA makes statistical inference difficult. The purpose of this study was to develop a hybrid ICA in the frequency domain that enables statistical inference while preserving advantages of a data-driven ICA. Three normal volunteers were scanned with fMRI while they performed a working memory task. Their data were analyzed with frequency domain hybrid ICA. In each of the subjects, the pat-terns of activation corresponded to areas expected to be active during the fMRI task. This investigation demonstrates that a hybrid ICA in the frequency domain can statistically map functional activation while preserving the ability of ICA to blindly separate noise sources from the data.
Kai Chun Chiu, Lei Xu
Tests of Gaussian
Temporal Factor Loadings in Financial APT
The main difficulty of financial APT analysis concerns identifying unambiguously the hidden statistical factors. Lack of effective techniques to retrieve the true factors often leads to inappropriate interpretation of the underlying factor structure. In literature, PCA and MLFA, assuming multivariate Gaussian distributions, and ICA, assuming non-Gaussian distributions, are used to extract factors and determine the corresponding factor loadings. Recently, a new technique called TFA is proposed in [1, 2] which seeks to solve the problem of rotation indeterminacy encountered in conventional factor analysis. In this paper we will focus on statistical tests and inference on the APT temporal factor loadings recovered by TFA.
Seungjin Choi, Andrzej Cichocki, Yannick Deville
Differential
Decorrelation for Nonstationary Source Separation
In this paper we consider the problem of source separation for the case that sources are (second-order) nonstationary, especially their variances are slowly time varying. The differential correlation is exploited in order to capture the time-varying statistics of signals. We show that nonstationary source separation can be achieved by differential decorrelation. Algebraic methods are presented and discussed.
Simone Fiori, Pietro Burrascano
Ect-Data Fusion by the Independent Component Analysis for Non-Destructive Evaluation of Metallic Slabs
The aim of this paper is to present an application of ICA to non-destructive evaluation by unsupervised data-fusion, which aims at discovering the flaws affecting a metallic slab. The signals acquired through an eddy-current probe for non-destructive evaluation purposes are affected by strong noise and disturbances due to the mechanical system that the probe is mounted on. The availability of multiple measurements allows performing a linear data-fusion which returns independent latent signals, one of which represents the flaw-related signal recovered from the noisy mixture. A one-unit neural ICA system is employed to extract such latent signal.
Pando Georgiev
Blind Source Separation of Bilinearly Mixed Signals
We propose a blind source separation model for statistically independent source signals, when the mixing operator is bi-linear. This model is equivalent to a linear model for separation of pairwise multiplications the source signals. We prove that if the source signals are are colored and have distinct autocorrelation functions on a given set P, then we can extract simultaneously the pairwise multiplications of them, using a generalized eigenvalue problem.
Gen Hori, Masato Inoue, Shin-ichi Nishimura, Hiroyuki Nakahara
Blind Gene
Classification Based on ICA of Microarray Data
The present study shows that an ICA-based method can, effectively and blindly, classify a vast amount of gene expression data into biologically meaningful groups. Specifically, we show (1) that genes, whose expression data are sampled at different times, can be classified into several groups, based on the correlation of each gene with independent component curves over time, and (2) that these classified groups by ICA-based method have a good match with the classified groups that are determined by use of domain knowledge and considered to be a benchmark. These results suggest that the ICA-based method can be a powerful approach to discover unknown gene functions.
Douglas R. Hundley, Michael J. Kirby, Markus G. Anderle
A Solution Procedure for Blind Signal Separation Using
the Maximum Noise Fraction Approach: Algorithms and Examples
In this paper, we outline the relationship between the Maxi-mum Noise Fraction (MNF) method–an algorithm first pro-posed for cleaning noise from multispectral image data
and Blind Signal Separation (BSS). In particular we demonstrate under what conditions these methods are equivalent and indicate that MNF may be viewed as an extension to
BSS for the case of subspace mixing. We present several examples and compare the results of the MNF method to algorithms for performing independent component analysis
(ICA).
Mika Inki, Aapo Hyvaerinen
Two Methods for Estimating Overcomplete Independent Component Bases
Estimating overcomplete ICA bases is a difficult problem that emerges when using ICA on many kinds of natural data, e.g. image data. Most algorithms are based on approximations of the likelihood, which leads to computationally heavy procedures. Here we introduce two algorithms that are based on heuristic approximations and estimate an ap-proximate overcomplete basis quite fast. The algorithms are based on quasi-orthogonality in high-dimensional spaces, and the gaussianization procedure, respectively.
Andreas Jung, Fabian J. Theis, Carlos G. Puntonet, Elmar W. Lang
Fastgeo – A Histogram
Based Approach to Linear Geometric ICA
Guided by the principles of neural geometric ICA, we present a new approach to linear geometric ICA based on histograms rather than basis vectors. Considering that the learning process converges to the medians and not the maxima of the underlying
distributions restricted to the receptive fields of the corresponding neurons, we observe a considerable improvement in separation quality of different distributions and a sizable reduction in computational cost by a factor of 100 at least. We further explore the accuracy of the algorithm depending on the number of samples and the choice of the mixing matrix. Finally we discuss the problem of high dimensions and how it can be treated with geometrical ICA algorithms.
Martin Kermit, Oliver Tomic
Independent
Components of Odour Signals
If two independent observations or processes are measured with the same apparatus, the inherent nature of the measuring device will in many cases introduce a dependency between the two recorded processes object to inspection. In this paper a suggestion of how Independent Component Analysis (ICA) can be used to identify such device dependencies and in turn give an estimated reconstruction of the observations without the correlation between signals introduced by the apparatus. The procedure is illustrated with the use of an "electronic nose" used to sample odours from mixtures of alcohol solutions. It is shown that ICA as a novel tool in the analysis of odour signals can extract the independent odour sources and give acceptable estimates of the ratio with which the alcohol solutions were mixed with two different approaches.
Mirko Knaak, Dieter Filbert
Acoustical Semi-Blind Source Separation for Machine Monitoring
In this contribution, convolutive blind source separation algorithms are compared with the well studied theory of minimum variance beamforming. As a result, the equivalence between the delay vector in the PCA (principle component analysis) subspace and the column the of rotation matrix belonging to the target sound is shown. That equivalence yields a new semi-blind algorithm being more robust than a minimum variance beamformer. The new algorithm is applied to the reconstruction of machine sounds (for classification purposes) when they are corrupted by strong interfering sources and a high noise level in a shop floor.
Mika Koganeyama, Hayaru Shouno, Toshyyasu Nagao, Kazuki Joe
Separation of Train Noise and Seismic Electric Signals from Telluric Current Data by ICA
Recently, detection of seismic electric signals (SESs) in telluric current data (TCD) observed using the VAN method has attracted notice for short-term earthquake prediction. However, since most of the TCD collected in Japan is affected by train noise, detecting SESs in TCD itself is an extremely arduous job. The goal of this research is to derive a method for detecting SESs, which is difficult for VAN method experts because of train noise. We believe that SESs and train noise are independent signals. Therefore we attempted to apply Independent Component Analysis (ICA) to several TCD data sets which were measured at Matsushiro, Nagano. As a result, train noise and SESs were successfully separated using ICA.
Dorothea Kolossa, Bert-Uwe Kohler, Markus Conrath, Reinhold Orglmeister
Optimal Permutation Correction by Multiobjective Genetic Algorithms
The separation of convolved signals can be performed in the frequency domain by splitting the signal into frequency bands, applying ICA techniques to each band and reassembling all bands to obtain the separated output signals. As ICA cannot guarantee for any ordering of output signals, the reassembly of the frequency bands must use some additional information in order to assign the frequency bands consistently to the right output. A number of different criteria have been proposed to solve this permutation problem. We have applied genetic algorithms at this point, which have the advantage of coping well with the discrete, multi-modal search space that characterizes this problem. Furthermore, applying a multiobjective genetic algorithm allows to take more than one criterion for optimal separation into account simultaneously. This can give an insight into the relative merit of various criteria of separation optimality.
Russell H. Lambert, Marcel Joho, Heinz Mathis
Polynomial Singular Values for Number of Wideband
Sources Estimation and Principal Component Analysis
A multipath enabled singular value decomposition (SVD) algorithm is presented, which will allow computation of wideband (polynomial) singular values, and hence, the signal+
noise and noise subspaces. Polynomial singular values are ordered according to total energy. The number of sources can be estimated using the scalar total energy values. Results using both simulated data on the computer and actual speech recorded in a noisy multipath environment are given to demonstrate the usefulness of the techniques
shown. After number of sources estimation, only the signal+ noise subspace is used to create virtual sensors which have made optimal use of all the sensors. As a final signal copy step, standard blind independent component analysis (ICA) or blind source separation algorithms can be used to recover the original data from the virtual sensors. The number of estimated sources could also be given to a blind algorithm capable of using overdetermined sources and the algorithm can adaptively make use of all sensor data.
Shun-Tian Lou, Xian-Da Zhang
Blind Source Separation for Changing Source Number: A Neural Network Approach with a Variable Structure
Blind source separation (BSS) problems have recently become an active research area in both statistical signal processing and unsupervised neural learning. In most approaches, the number of source signals is typically assumed to be known a priori, but this does not usually hold in practical applications. Although the problem of determining the unknown source number has been studied recently, the BSS problem when the source number is changing dynamically is not yet considered. The main objective of this paper is to study and solve these two problems. Its basic idea is to utilize the correlation coefficients between output components of the neural network (NN) as a mean for determining the unknown source number and/or detecting dynamical change of the source number, and is to develop a neural network with variable structure to perform the corresponding adaptive blind source separation.
Ruben MartinClemente, Carlos G. Puntonet, Jose I. Acha
Blind Signal Separation Based on the Derivatives of the Output Cumulants and a Conjugate Gradient Algorithm
In this paper it is proven that the estimation of the separating system can be based on the cancellation of some second partial derivatives of the output cross-cumulants. We propose a new contrast function that must be optimized on the Stiefel manifold. A conjugate gradient method is used in order to obtain a fast convergence speed.
Hyung-Min Park, Sang-Hoon Oh, Soo-Young Lee
On Adaptive Noise Canceling Based on Independent Component Analysis
We present a method to deal with adaptive noise canceling based on independent component analysis (ICA). Although popular least-mean-squares (LMS) algorithm removes noise components based on second-order correlation, the proposed ICA-based algorithm can utilize higher-order statistics. Additionally, extending to transform-domain adaptive filtering (TDAF) methods, normalized ICA-based algorithm is de-rived to improve convergence rates. Experimental results show that the proposed ICA-based algorithm provides much better performances than conventional LMS approach in real world problems.
F.Rojas, I.Rojas , R.M.Clemente , C.G.Puntonet
Nonlinear Blind
Source Separation Using Genetic Algorithms
This article proposes the fusion of two important paradigms, Genetic Algorithms and the Blind Separation of Sources (GABSS). Although the topic of BSS, by means of various techniques, including ICA, PCA and neural networks, has been amply discussed in the literature, to date the possibility of using genetic algorithms has not been explored. However, in nonlinear mixtures, optimisation of the system parameters and, especially, the search for invertible functions is very difficult by the existence of many local minima. From experimental results, this paper demonstrates the possible benefits offered by Gas I in combination with BSS, such as robustness against local minima, the parallel search for various solutions, and a high degree of flexibility in the evaluation function.
Justinian Rosca, NingPing Fan, Radu Balan
Real-Time Audio Source Separation by Delay and Attenuation Compensation in the Time Domain
There is increased interest in using microphone arrays in a variety of audio source separation and consequently speech processing applications. In particular, small arrays of two to four microphones are presently under focus in the research literature, especially with regard to real-time source separation and speech enhancement capability. In this paper we focus on a real-time implementation of the delay and attenuation compensation (DAC) algorithm. Although the algorithm is designed for anechoic environments, its complexity and performance on real data represent a basis for designing more complex approaches to deal with reverberant environments. We highlight real-time issues and
analyze the algorithm’s real-time performance on a database of more than 1000 mixtures of real voice recordings ranging from an anechoic to a strongly echoic office with reverberation time of 500 msec.
Hiroshi Saruwatari, Toshiya Kawamura, and Kiyohiro Shikano
Blind Source
Separation Based on Fast-Convergence Algorithm Using ICA and Array Signal
Processing
We propose a new algorithm for blind source separation (BSS), in which independent component analysis (ICA) and beamforming are combined to resolve the low convergence problem through optimization in ICA. The proposed method consists of the following three parts: (1) frequency-domain ICA with direction-of-arrival (DOA) estimation, (2) null beamforming based on the estimated DOA, and (3) integration of (1) and (2) based on the algorithm diversity in both iteration and frequency domain. The inverse of the mixing matrix obtained by ICA is temporally substituted by the matrix based on null beamforming through iterative optimization, and the temporal alternation between ICA and beamforming can realize fast- and high-convergence optimization. The results of the signal separation experiments reveal that the signal separation performance of the proposed algorithm is superior to that of the conventional ICA-based BSS method, even under reverberant conditions.
Fabian J. Theis, Andreas Jung, Elmar W. Lang, Carlos G. Puntonet
A Theoretic Model for
Linear Geometric ICA
Geometric algorithms for linear ICA have recently received some attention due to their pictorial description and their relative ease of implementation. The geometric approach to ICA has been proposed first by Puntonet and Prieto [1] [2] in order to separate linear mixtures. We will reconsider geometric ICA in a solid theoretic framework showing that fix-points of geometric ICA fulfill a so called geometric convergence condition, which the mixed images of the unit vectors satisfy, too. This leads to a conjecture claiming that in the supergaussian unimodal symmetric case there is only one stable fixpoint, thus demonstrating uniqueness of geometric ICA after convergence.6
Ana Maria Tomé
An Iterative
Eigendecomposition Approach to Blind Source Separation
In this work we address the generalized eigendecomposition approach (GED) to the blind source separation problem. We present an alternative formulation for GED based on the definition of congruent pencils. Making use of this definition, and matrix block operations, the eigendecompostion approach to blind source separation is completely characterized. We also present an iterative method to compute the eigendecomposition of a symmetric positive definite pencil.
Oliver Tomic, Martin Kermit
Discrimination and
Interpretation of Electronic Nose Data Using ICA
This work reports on independent component analysis (ICA) as a tool used to discriminate between odour signals. Measurements of six different alcohol solutions were carried out with a commercial gas sensor array system, a so called electronic nose. The solutions were made of either pure propanol or butanol at concentration levels of 0.5%, 1% and 2%. Principal component analysis (PCA), a standard multivariate analysis tool for gas sensor data, needed three principal components (PC) for effective discrimination of the solutions. With ICA, only two independent components (IC) were needed to achieve a similar result. PC-1 and IC-1 gave both meaningful representations of alcohol concentrations in the solutions. However, only a combination of PC2 and PC3 could represent different types of alcohols as effectively as IC2 did.
Andreas Ziehe, Motoaki Kawanabe, Stefan Harmeling, KlausRobert Muller
Separation of Post-Nonlinear Mixtures Using Ace and
Temporal Decorrelation
We propose an efficient method based on the concept of maximal correlation that reduces the post-nonlinear blind source separation problem (PNL BSS) to a linear BSS problem. For this we apply the Alternating Conditional Expectation (ACE) algorithm – a powerful technique from non-parametric statistics – to approximately invert the (post)non-linear functions. Interestingly, in the framework of the ACE method convergence can be proven and in the PNL BSS scenario the optimal transformation found by ACE will coincide with the desired inverse functions. After the non-linearities have been removed by ACE, temporal decorrelation (TD) allows us to recover the source signals. An excellent performance underlines the validity of our approach and demonstrates the ACE-TD method on realistic examples.
.
Pedro Hojen-Sorensen, Lars Kai Hansen, Ole Winther
Mean Field
Implementation of Bayesian ICA
In this contribution we review the mean field approach to Bayesian independent component analysis (ICA) recently developed by the authors [1, 2]. For the chosen setting of additive Gaussian noise on the measured signal and Maximum Likelihood II estimation of the mixing matrix and the noise, the expected sufficient statistics are obtained from the two first posterior moments of the sources. These can be effectively estimated using variational mean field theory and its linear response correction. We give an application to feature extraction in neuro-imaging using a binary (stimuli/no stimuli) source paradigm. Finally, we discuss the possibilities of extending the framework to convolutive mixtures, temporal and ‘spatial’ source prior correlations, identification of common sources in mixtures of different media and ICA for density estimation.
Jianting Cao, Noboru Murata, Shunichi Amari, Andrzej Cichocki, Tsunehiro Takeda
A Robust ICA Approach for Unaveraged Single-Trial Auditory Evoked Fields Data Decomposition
Treating an averaged EFs (evoked-fields) or ERPs (event-related potentials) data is a main approach in recent topics on applying ICA to neurobiological signal processing. By
taking the average, the signal-noise ratio (SNR) is increased but some important information such as the strength of an evoked response and its dynamics will be lost. The single-trial data analysis, on the other hand, can avoid this problem but the poor SNR arises. In this paper, we present a robust approach for decomposing and localizing unaveraged single-trial MEGdata. Our approach has two procedures. In the first step, a PCA-like pre-whitening with the high-level noise reduction and an optimal dimensionality reduction techniques are presented. In the second step, a robust nonlinear function derived by the parameterized t-distribution model is applied to separate the mixtures of sub-Gaussian and super-Gaussian source components. The results on unaveraged AEFs single-trial data analysis illustrate that not only the behavior and location but also the activity strength (amplitude) and dynamics of the individual evoked response can be visualized by using the proposed method.
_
_
Jaakko Särelä, Harri Valpola, Ricardo Vigário, and Erkki Oja
Dynamical Factor Analysis of Rhythmic Magnetoencephalographic Activity
Dynamical factor analysis (DFA) is a generative dynamical algorithm, with linear mapping from factors to the observations and nonlinear mapping of the factor dynamics. The latter is modeled by a multilayer perceptron. Ensemble learning is used to estimate the DFA model in an unsupervised manner. The performance of the DFA have been
tested in a set of artificially generated noisy modulated si-nusoids. Furthermore, we have applied it to magnetoencephalographic data containing bursts of oscillatory brain activity. This paper shows that DFA can correctly estimate the underlying factors in both data sets.
A.Delorme, S. Makeig, T. Sejnowski
Automatic Artifact Rejection in EEG Using High Order Statistics and Independent Component Analysis
While it is now generally accepted that independent component analysis is a good tool for isolating both artifacts and cognitive related activations in EEG data, there is still little consensus about criteria for automatic rejection of artifactual components and single trials. Here we developed a graphical software to semi-automatically assist experimenter in rejecting independent components and noisy single data trials based on their statistical properties. We used kurtosis to detect peaky activity distributions that are characteristic of some types of artifact and entropy to detect unusual activity patterns. EEG-LAB, a user-friendly graphic interface running under Matlab, allows the user to tune and calibrate the rejection criteria, to accept or override the suggested components and trials labeled for rejection, and to compare the results with other rejection methods.
Kevin H. Knuth, Wilson A. Truccolo, Steven L. Bressler, Mingzhou Ding
Separation of Multiple Evoked Responses using Differential Amplitude and Latency Variability
In neuroelectrophysiology one records electric potentials or magnetic fields generated by ensembles of synchronously active neurons in response to externally presented stimuli. These evoked responses are often produced by multiple generators in the presence of ongoing background activity. While source localization techniques or current source density estimation are usually used to identify generators, application of blind source separation techniques to obtain independent components has become more popular. We approach this problem by applying the Bayesian methodology to a more physiologically-realistic source model. As it is generally accepted that single trials vary in amplitude and latency, we incorporate this variability into the model. Rather than making the unrealistic assumption that these cortical components are independent of one another, our algorithm utilizes the differential amplitude and latency variability of the evoked waveforms to identify the cortical components. The algorithm is applied to intracortically-recorded local field potentials in monkeys performing a visuomotor task.
Jagath C. Rajapakse, Wei Lu
Extracting Task-Related Components in Functional MRI
To extract consistently and transiently task-related component maps, a novel ICA paradigm, the ICA with reference (ICA-R), is proposed. ICA-R produces only component maps corresponding to the input stimuli which are used as reference signals in the learning paradigm, and all activations corresponding to a particular stimulus are located in a single component map. Computational and memory requirements of ICA-R are much less than those required by spatial ICA or temporal ICA.
Irina F. Gorodnitsky , Adel Belouchrani
Joint Cumulant and Correlation Based Signal
Separation with Application to EEG Data Analysis
Current methods in Blind Source Separation (BSS) utilize either the higher order statistics or the time delayed cross-correlations to perform signal separation. In this paper we investigate a method for source separation which utilizes joint information from higher order statistics and delayed cross-correlations. The algorithm is motivated by problems in analysis of Electroencephalography (EEG) data. We use an EEG analysis example to demonstrate that the Joint Cumulant and Correlation based (JCC) algorithm obtains better source separation than either of the group methods based on higher order statistics or time delayed cross correlations.
Diffusion Networks, Product of Experts, and Factor Analysis
Hinton (in press) recently proposed a learning algorithm called contrastive divergence learning for a class of probabilistic models called product of experts (PoE). Whereas in standard mixture models the “beliefs” of individual experts are averaged, in PoEs the “beliefs” are multiplied together and then renormalized. One advantage of this approach
is that the combined beliefs can be much sharper than the individual beliefs of each expert. It has been shown that a restricted version of the Boltzmann machine, in which
there are no lateral connections between hidden units or between observation units, is a PoE. In this paper we generalize these results to diffusion networks, a continuous-time,
continuous-state version of the Boltzmann machine. We show that when the unit activation functions are linear, this PoE architecture is equivalent to a factor analyzer. This result suggests novel non-linear generalizations of factor analysis and independent component analysis that could be implemented using interactive neural circuitry.
Laurent Giulieri, Nadège Thirion-Moreau and Pierre-Yves Arquès
Blind Source Separation Using Bilinear and Quadratic Time-Frequency Representations
We consider here the problem of blind sources separation. During the last decade, many solutions have been proposed among which contrasts functions, maximum likelihood functions, information-theoretic criteria, etc... More recently, a new method based on some time-frequency (t-f ) representations has been introduced by Belouchrani et al. It consists in joint-diagonalizing a combined set of “spatial t-f distributions (stfd)” matrices. However, t-f representations properties still not have been widely exploited to solve the sources separation problem. Our aim is to develop this point to take better advantage of bilinear and quadratic time-frequency representations properties. Hence, we derive new criteria of choice of stfd matrices sets to be joint-diagonalized and/or joint anti-diagonalized. Finally, some computer simulations are presented in order to demonstrate the effectiveness of the proposed algorithm.
Kwokleung Chan, TeWon Lee, Terrence Sejnowski
Variational Learning of Clusters of Undercomplete Nonsymmetric Independent Components
We apply a variational method to automatically determine the number of mixtures of independent components in high-dimensional datasets, in which the sources may be non-symmetrically distributed. The data is modeled by clusters where each cluster is described as a linear mixing of independent factors. Because of the variational bayesian treatment, this method can yield an accurate density model for the observed data without overfitting problems. This allows us also to identify the dimensionality of the data for each cluster. The new method is applied to a difficult real-world medical dataset and is successful in diagnosing glaucoma.
Sepp Hochreiter, Michael C. Mozer
Monaural Separation and Classification of Mixed Signals: A Support-Vector Regression Perspective
We address the problem of extracting multiple independent sources from a single mixture signal. Standard independent-component analysis approaches fail when the number of sources is greater than the number of mixtures. For this case, the sparse-decomposition method [1] has been proposed. The method relies on a dictionary of atomic signals and recovers the degree to which various dictionary atoms are present in the mixture. We show that the sparse-decomposition method is equivalent to a form of support-vector regression (SVR). The training inputs for the SVR are the dictionary atoms, and the corresponding targets are the dot product of the mixture and atom vectors. The SVR perspective provides a new interpretation of the sparse-decomposition method’s hyperparameter, and allows us to generalize and improve the method. The most important insight is that the sources do not have to be identical to dictionary atoms, but rather we can accommodate a many-to-one mapping of source signals to dictionary atoms—a classification of sorts—characterized by a known nonlinear transformation with unknown parameters. The limitation of the SVR perspective is that it cannot recover the signal strength of an atom in the mixture; rather, it can only recover whether or not a particular atom was present. In experiments, we show that our model can handle difficult problems involving classification of sources. Our model may be particularly useful for speech signal processing and CDMA-based mobile communication, where in both cases we have knowledge about the invariances in the signal.
Pavel Kisilev, Michael Zibulevsky, Yehoshua Y. Zeevi, Barak A. Pearlmutter
Blind Source Separation via Multinode Sparse
Representation
The blind source separation problem is concerned with extraction of the underlying source signals from a set of their linear mixtures, where the mixing matrix is unknown. It was discovered recently, that exploiting the sparsity of sources in an appropriate representation according to some signal dictionary, dramatically improces the quality of separation. In this work we use the property of multiscale transforms, such as wavelet or wavelet packets, to decompose signals into sets of local features with various degrees of sparsity. We use this intrinsic property for selecting the best (most sparse) subsets of features for further separation. The performance of the algorithm is vertified on noise-free and noisy data. Experiments with simulated signals, musical sounds and images demonstrate significant improvement of separation quality over previously reported results.
Juan K. Lin
Lattice Decompositions of Multivariate Probability Density Functions Extending the ICA Model to Incorporate More General Source Structures and Dependencies
The classical ICA model assumes that observations are all linear combinations of statistically independent scalar sources. This, as well as prior assumptions on the number of sources and their distributions are often seen as the weakest aspects of the ICA source model. In this paper, we present the mathematical structure necessary for extending ICA to more flexible models of real-world data.
Mark Plumbley
Adaptive Lateral Inhibition for Non-Negative ICA
We consider the problem of decomposing an observed input matrix (or vector sequence) X into the product of a mixing matrix W with a component matrix Y, i.e.X = WY, where (a) the elements of the mixing matrix and the component matrix are non-negative, and (b) the underlying components are considered to be observations from an independent source. This is therefore a problem of non-negative independent component analysis. Under certain reasonable conditions, it appears to be sufficient simply to ensure that the output matrix has diagonal covariance (in addition to the non-negativity constraints) to find the independent basis. Neither higher-order statistics nor temporal correlations are required. The solution is implemented as a neural net-work with error-correcting forward/backward weights and linear anti-Hebbian lateral inhibition, and is demonstrated on small artificial data sets including a linear version of the Bars problem.
Arie Yeredor
Blind Source Separation with Pure Delay Mixtures
We address the problem of blind
separation of mixtures consisting of pure unknown delays in addition to scalar
mixing coefficients. Such a mixture is a hybrid situation resembling both
static and convolutive mixtures, but essentially different from both: On one
hand, static-mixture approaches cannot be readily applied in this context; On
the other hand, conventional convolutive-mixture approaches are not only over -
parameterized for this problem, but are also incapable of accommodating
fractional delays. We propose a second-order statistics based algorithm, which
uses a specially parameterized approximate joint diagonalization of spectral
matrices to estimate the mixing coefficients as well as the delays. The joint
diagonalization algorithm is an extension of the "AC-DC" algorithm,
previously proposed in the context of static mixtures. We provide analytic expressions for all
minimization steps for the two sensors/ two sources case, and demonstrate the
performance using simulations results.
J.-R. Duann, T.-P. Jung, W.-J. Kuo, T.-C. Yeh, S. Makeig, J.-C. Hsieh, T. J. Sejnowski
Measuring the Variability of Event-Related Bold Signal
Most current analysis methods for functional magnetic resonance imaging (fMRI) data assume a priori knowledge of the time course of the hemodynamic response (HR) to experimental stimuli or events in brain areas of interest. In addition, they typically assume homogeneity of both the HR and the non-HR “noise” signals both across brain regions and across similar experimental events. When HRs vary unpredictably
from area to area, or from trial to trial, different approaches are needed. Here we used infomax Independent Component Analysis (ICA) to detect and visualize variations in single-trial HRs in event-related fMRI data. ICA decomposition of the resulting BOLD data produced independent components with variable stimulus-locked HRs active in primary visual (V1) and medial temporal (MT/V5) cortices respectively. Contrary to
expectation, in four of six subjects the HR of the V1 component contained two positive peaks in response to short-stimulus bursts, while nearly identical component maps were associated with single-peaked HRs in long-stimulus sessions from the same subject. Thus, ICA combined with single-trial visualization can reveal dramatic and unforeseen task-related HR variation not apparent to researchers analyzing the data with fixed HR
templates.
Samer A. Abdallah, Mark D. Plumbley
If the Independent Components of Natural Images are Edges, What are the Independent Components of Natural Sounds?
Previous work has shown that various flavours of Independent Component Analysis, when applied to natural images, all result in broadly similar localised, oriented band-pass feature detectors, which have been likened to wavelets or edge detectors.
In this paper, we present a similar analysis of ‘natural’ sounds drawn from two radio stations: one broadcasting mainly speech; the other mainly classical music. Many of
the resulting basis vectors are quite wavelet-like, and can easily be characterised in terms of their position and spread in the time-frequency plane. Some of them, however, particularly from the set trained on music, do not fit that inter-pretation very well. The Wigner-Ville Distribution can be used to gain a clearer picture of time-frequency localization of these basis vectors. We conclude by suggesting that these results be com-pared with other widely used auditory representations such as short-term Fourier transforms, wavelet transforms, and physiologically derived models based on the auditory filter-bank.
Thomas Kolenda, Lars Kai Hansen, Jan Larsen
Signal Detection Using ICA: Application to Chat Room Topic Spotting
Signal detection and pattern recognition for online grouping huge amounts of data and retrospective analysis is becoming increasingly important as knowledge based standards, such as XML and advanced MPEG, gain popularity. Independent component analy-sis
(ICA) can be used to both cluster and detect signals with weak a priori assumptions in multimedia contexts. ICA of real world data is typically performed without knowledge of the number of non-trivial independent components, hence, it is of interest to test hypotheses concerning the number of components or simply to test whether a given set of components is significant relative to a “white noise” null hypothesis. It was recently proposed to use the so-called Bayesian information criterion (BIC) approximation, for estimation of such probabilities of competing hypotheses. Here, we apply this approach to the understanding of chat. We show that ICA can detect meaningful context structures in a chat room log file.
Ella Bingham
Topic Identification in Dynamical Text by Extracting Minimum Complexity Time Components
The problem of analysing dynamically evolving textual data has recently arisen. An example of such data is the discussion appearing in Internet chat lines. In this paper a recently introduced source separation method, termed as complexity pursuit, is applied to the problem. The method is a generalisation of projection pursuit to time series and it is able to use both spatial and temporal dependency information in separating the topics of the discussion. Experimental results on chat line and newsgroup data demonstrate that the minimum complexity time series indeed do correspond to meaningful topics inherent in the dynamical text data, and also suggest the applicability of the method to query-based retrieval from a temporally changing text stream. The complexity pursuit method is compared to several ICA-type algorithms for time series.
Johan Himberg, Aapo Hyvaerinen
Independent Component Analysis for Binary Data: An Experimental Study
We consider a mixing model where independent binary components are mixed using binary OR operations. Using extensive simulations, we investigate whether the model can be estimated using ordinary cumulant-based ICA algorithms. We show that the model can indeed be estimated if the data is sparse enough. We also compare the 3rd and 4the order cumulants. In the no-noise and low-noise cases, the 3rd order cumulant performs better, but in the presence of strong noise, the 4th –order cumulant, somewhat surprisingly, performs better for very sparse data.
Yu-Hwan Kim, Byoung-Tak Zhang
Document Indexing Using Independent Topic Extraction
Text retrieval involves finding relevant information from a collection of documents given the user’s information need. Traditional information retrieval systems represent a document as a vector of words, where each component can be a simple word count or follows a more sophisticated weighting scheme. The composed matrix is usually sparse and some words are highly correlated with each other. Another representation scheme is focused on the underlying topics, which is usually obtained by a variety of dimension reduction techniques, where each document can be represented as a vector of topic intensity. In this paper, We proposed a novel indexing technique based on independent component analysis. From the experiments performed on AP news articles, the performance improvements is significant when the topic of query is closely related to the topics which is extracted from the ICA.
Wakako Hashimoto
Independent Component Analysis with Several Mixing Matrices
A new algorithm is proposed for the variation of independent component analysis (ICA) in which there are several mixing matrices and, for each set of independent components,
one of the matrices is randomly chosen to mix the components. The algorithm utilizes high-order moments and can obtain consistent estimators even if the true probability density function of independent components is not obtained. The effectiveness of our algorithm is verified by a numerical experiment. This method can be used to analyze a class of data generated overcompletely, and to classify data in an unsupervised manner.
Adriana Dapena, Christine Serviere
A Simplified Frequency-Domain Approach for Blind Separation of Convolutive Mixtures
In the frequency-domain, a convolutive mixture can be interpreted as several instantaneous mixtures which may be separated using many existing algorithms. The main limitation of this approach is the large number of frequencies that must be considered. In addition, a permutation/amplitude correction must be performed when the sources are recovered in a different order or with different amplitudes in some frequency bins. In this paper we propose a novel approach for solving the convolutive problem using only two frequency bins. The idea deals with finding a 2 X 2 invertible matrix
which relates the signals recovered in these frequency bins and the temporal sources. The existence of this matrix is due to use zero-padding for computing the Fourier transform of the observations. Several simulation results show the good performance of this simplified approach in different applications.
Simone Fiori, Pietro Burrascano
Electromagnetic Environmental Pollution Monitoring: Source Localization by the Independent Component Analysis
The aim of this paper is to present an electromagnetic source localization technique based on independent component analysis (ICA). Two ICA algorithms known from the literature, allowing to process complex-valued signals, are used to es-timate the mixing operator from electromagnetic data; as the mixing operator contains important information about the source complex structure and about the electromagnetic field propagation phenomena, by properly interpreting the results given by the ICA algorithm it is possible to develop a blind source localization procedure. Performing such
procedure is the first step in electromagnetic environmental pollution monitoring.
Pando Georgiev, Andrzej Cichocki
On Some Extensions of Natural Gradient Algorithm
Recently several novel gradient descent approaches like natural or relative gradient methods have been proposed to derive rigorously various powerful ICA algorithms. In this paper we propose some extensions of Amari’s Natural Gradient and Atick-Redlich formulas. They allow us to derive rigorously some already known algorithms, like for example, robust ICA algorithm and local algorithm for blind decorrelation. Furthermore, we hope they enable us to generate the family of new algorithms with improved convergence speed or performance for various applications. We present conditions for which the proposed general gradient descent dynamical systems are stable. We show that the nonholonomic orthogonal algorithm can not be derived from minimization of any cost function. We propose a stabilized nonholonomic algorithm, which preserves the norm of the demixing matrix.
Shahram Hosseini, Christian Jutten, Dinh Tuan Pham
Blind Separation of Temporally Correlated Sources Using a Quasi Maximum Likelihood Approach
A quasi-maximum likelihood approach is used for separating the instantaneous mixtures of temporally correlated, independent sources without either any preliminary transformation or a priori assumption about the probability distribution of the sources. A first order Markov model is used to represent the joint probability density of successive samples of each source. The joint probability density functions are estimated from the observations using a kernel method.
Jorge Igual, Andrés Camacho, Luis Vergara
Canceling Sinusoidal Interferences in Ultrasonic Applications
with a BSS Algorithm for More Sources than Sensors
Sinusoidal interferences are found in ultrasonic signals when we try to characterize a material, as for example interferences coming from PC cards. We are interested in obtain a robust method that cancels these interferences preserving the waveform of the signal. A Blind Source Separation BSS method to extract these sinusoids is presented in this paper. We will get so many linear mixtures of the backscattering echo of the material and the sinusoids as we need from different pulse responses of the material.
Gil-Jin Jang, Te-Won Lee, Yung-Hwan Oh [109]
Blind Separation of Single Channel Mixture Using ICA Basis Functions
A new technique has been developed to enable blind source separation given only a single channel recording. The proposed method infers source signals and their contribution factors at each time point by a number of adaptation steps maximizing log-likelihood of the estimated source parameters given the observed single channel data and sets of basis functions. This inferencing is possible due to the prior information on the
inherent time structure of the sound sources by learning a priori sets of time-domain basis functions and the associated coeÆcient densities that encode the sources in a statistically efficient manner. A model for density estimation allows accurate modeling of the observation and our experimental results show close- to-perfect separation on simulated mixtures as well as recordings in a real environment employing mixtures of two different sources.
Inseon Jang, Seungjin Choi
Sequential Least Squares Algorithms for Blind Co-Channel Signal Separation
In this paper we consider a problem of blind co-channel signal separation, the goal of which is to estimate multiple co-channel digitally modulated signals using an antenna array. We consider the joint maximum likelihood estimation [1] and present a sequential algorithm, which is referred to as sequential joint maximum likeli-hood (SJML) algorithm. In addition we also apply the sequential least squares to ILSP [2] and the resulting algorithm is referred to as the sequential least squares with projection (SLSP). Useful behavior of these two algorithms are confirmed by simulations.
M. Klajman, A.G. Constantinides [139]
A Combined Statistics Cost Function for Blind and Semi-Blind Source Separation
Most Blind Source Separation algorithms separate the sources by using either second or higher order statistics. In this paper we suggest to use a weighted combination of the second order covariance matrices and the fourth order eigenmatrices to restore the original algorithms relying on a single type of statistic only, fail. We then develop a novel diagonalisation algorithm, the Cayley Joint Unitary Diagonalisation (CJUD) algorithm, to find the optimal unitary diagonaliser and to determine the weights. Except in some trivial cases, it is very hard to determine the weights without any prior information. We show in this paper how the use of some prior knowledge can be incorporated in the CJUD algorithm in order to get a better estimate of the weights. Simulations are presented to show the improvement in performance of the proposed algorithms.
Jos Koetsier, Donald MacDonald, Darryl Charles
Exploratory Correlation Analysis
We present a novel unsupervised artificial neural network for the extraction of common features in multiple data sources. This algorithm, which we name Exploratory Correlation Anal-ysis (ECA), is a multi-stream extension of a neural implementation
of Exploratory Projection Pursuit (EPP) and has a close relationship with Canonical Correlation Analysis (CCA). Whereas EPP identifies ”interesting” statistical directions in a single stream of data, ECA develops a joint coding of the common underlying statistical features across a number of data streams. It has been shown that the principle of contextual guidance may be used to find a sparse coding of the features in dual natural image patches that is very different from single stream sparse coding experiments. The network only identifies those features which exist in both data streams and thus tend to be fewer in number and more complex in nature.
Ivica Kopriva , Harold Szu
Huygens-Fresnel Principle Generates Nonlinear ICA Reducible to Solvable Incoherent Limit Linear ICA
Reticle trackers have been used successfully with a beam splitter for tracking and discrimination of several moving incoherent (heat) optical sources in the mathematical framework called Independent Component Analyses (ICA). Here we further explore the theoretical basis of the coherent and partially coherent illumination by laser for the possibility of blind source de-mixing. An application of the partial coherence theory and the Hutgens-Fresnel principle is utilized to formulate the problem. When incoherence is assumed a linear ICA model is obtained while in most general case of either partially or totally coherent optical radiation the resulting signal model is inherently nonlinear. It can be transformed into linear one under very special condition that assumes no relative motion between the radiating sources. In the most general case of partially coherent radiation, tracking of the several moving optical sources by using the beam splitter based reticle trackers is possible either by using ICA algorithms developed for undercomplete representation or by introduction of one additional sensor.
Hervé Le Borgne, Anne Guérin-Dugué
Sparse-Dispersed Coding and Images Discrimination with Independent Component Analysis
Independent Component Analysis applied to a set of natural images provides band-pass-oriented filters, similar to simple cells of the primary visual cortex. We applied two types of pre-processing to the images, a low-pass and a whitening one in a multiresolution grid, and examine the properties of the detectors extracted by ICA. These detectors composed a new basis function set in which images are encoded. On one hand, the properties (sparseness and dispersal) of the resulting coding are compared for both pre-processing strategies. On the other hand, this new coding by independent features is used for discriminating natural images, that is a very challenging domain in image analysis and retrieval. We show that a criterion based on the dispersal property enhances the efficiency of the discrimination by selecting the most dispersed detectors coding the image database. This behaviour is well enhanced with whitened images.
Jae Sung Lee, Daniel D. Lee, Seugjin Choi, Dong Soo Lee
Application of Non-Negative Matrix Factorization to Dynamic Positron Emission Tomography
Recently suggested non-negative matrix factorization (NMF) seems to overcome fundamental limitations of factor analysis at least in theoretical aspect. NMF cost function uses Poisson statistics as a noise model, rather than the Gaussian statistics, and provides a simple learning rule, in contrast to the tricky optimization in factor analysis. To study the feasibility of NMF for the analysis of dynamic image sequences in nuclear medicine, NMF was applied to H2 15 O dynamic myocardial PET images acquired from dog studies, and the results were compared with those obtained by conventional factor analysis method. Using NMF we could obtain basis images corresponding to major cardiac components. Their time-activity curves showed reasonable shapes that we have been familiar with. With the assumption of proper number of factors, NMF presented good results at least similar with those by factor analysis. Our results showed that NMF would be feasible for image segmentation and factor extraction from dynamic image sequences in nuclear medicine.
N. Mitianoudis, M. Davies
New Fixed-Point ICA Algorithms for Convolved Mixtures
One of the most powerful techniques applied to blind audio source separation is Independent Component Analysis (ICA). For the separation of audio sources recorded in a real environment, we need to model the mixing process as convolutional. Many methods have been introduced for separating convolved mixtures, the most successful of which require working in the frequency domain [1], [2], [3], [4]. Most of these methods perform efficient separation of convolved mixtures, however they are relatively slow. The authors propose two fixed-point algorithms for performing fast frequency domain ICA.
Satoshi Niijima, Shoogo Ueno
Universal Fourth Order Music Method: Incorporation of ICA into Meg Inverse Solution
In recent years, several inverse solutions of magnetoenceph-alography (MEG) have been proposed. Among them, the multiple signal classification (MUSIC) method utilizes spatio-temporal information obtained from magnetic fields. The conventional MUSIC method is, however, sensitive to Gaussian noise and a sufficiently large signal-to-noise ratio (SNR) is required to estimate the number of sources and to specify the precise locations of electrical neural activities. In this paper, a universal fourth order MUSIC (UFO-MUSIC) method, which is based on fourth order statistics, is proposed. This method is shown to be more robust against Gaussian noise than the conventional MUSIC method. It is an algebraic approach to independent component analysis (ICA). Although ICA and the analysis of the MEG inverse problem have been separately discussed, the proposed method incorporates ICA into the MEG inverse solution. The results of numerical simulations demonstrate the validity of the proposed method.
Danielle Nuzillard, Albert Bijaoui
Multiscale Contrasts for Blind Source Separation
The analysis of multispectral astronomical images was examined as a blind source separation (BSS) of an instantaneous mixture. The experimented BSS methods were based on different assumptions: i/the non-Gaussianity measured by high order statistics (HOS), ii/ the correlation between shifted sources. A HOS approach taking into account the spatial organization is proposed from a decomposition in the wavelet space. A multiscale contrast taking into account the probability density function (PDF) at each scale, was first introduced. The resulting contrast provides few improvement in the separation. Thus a definition of the multiscale contrast which took into account a mask associated to a thresholding was examined. We show in this communication that this new statistical quantity provides an available criterion for BSS. .
Scott Rickard, Radu Balan, Justinian Rosca
Real-Time Time-Frequency Based Blind Source Separation
We present a real-time version of the DUET algorithm for the blind separation of any number of sources using only two mixtures. The method applies when sources are W-disjoint orthogonal, that is, when the supports of the win-dowed Fourier transform of any two signals in the mixture are disjoint sets, an assumption which is justified in the Appendix. The online algorithm is a Maximum Likelihood (ML) based gradient search method that is used to track the mixing parameters. The estimates of the mixing parameters are then used to partition the time-frequency representation of the mixtures to recover the original sources. The technique is valid even in the case when the number of sources is larger than the number of mixtures. The method was tested on mixtures generated from different voices and noises recorded from varying angles in both anechoic and echoic rooms. In total, over 1500 mixtures were tested. The average SNR gain of the demixing was 15 dB for anechoic room mixtures and 5 dB for echoic office mixtures. The algorithm runs 5 times faster than real time on a 750MHz laptop computer. Sample sound files can be found here: http://www.princeton.edu/˜srickard/bss.html
T.H. Sander, G. Wubbeler, L. Trahms, A. Lueschow, G. Curio
Identification of Visually Evoked Brain Activity and Cardiac Artifact Components through Time-Delayed Decorrelation
Applying the time-delayed decorrelation (TDD) algorithm to raw data from a visual stimulation magnetoencephalo-graphic (MEG) experiment we investigate the viability of
classification of components into artifact or stimulus related components. The TDD components associated with the cardiac artifact show a striking similarity with a Principal Component Analysis (PCA) of the averaged cardiac artifact. This could be due to a violation of the TDD assumptions by the cardiac artifact. Two TDD components have time series peaking consistently at the time expected for the primary response due to the visual stimulation, but their field maps are different from the earliest signal in the average response. An identification of TDD components with physiological sources needs further investigation.
Hiroshi Sawada, Ryo Mukai, Shoko Araki, Shoji Makino
A Polar-Coordinate Based Activation Function for Frequency Domain Blind Source Separation
This paper presents a new activation function for an ICA algorithm to process complex-valued signals, which is used in frequency domain blind source separation. The new activation function is based on the polar coordinates of a complex number, whereas the conventional one is based on the Cartesian coordinates of a complex number and calculates the real part and imaginary part separately. The new activation function eliminates an undesirable constraint occurred by the conventional function. In experiments for separating speech signals in a reverberant environment, we obtained improved SNRs by using the new activation function.
Fabian J. Theis, Elmar W. Lang
Maximum Entropy and Minimal Mutual Information in a Nonlinear Model
In blind source separation, two different separation techniques are mainly used: Minimal Mutual Information (MMI), where minimization of the mutual output information yields an independent random vector, and Maximum Entropy (ME), where the output entropy is maximized. However, it is yet unclear why ME should solve the separation problem, ie.
result in an independent vector. Amari has given a partial confirmation for ME in the linear case in [1], where he proves that under the assumption of vanishing expectancy of the sources ME does not change the solutions of MMI up to scaling and permutation.
In this paper, we generalize Amari’s approach to nonlin-ear ICA problems, where random vectors have been mixed by output functions of layered neural networks. We show that certain solution points of MMI are kept fixed by ME if no scaling of the weight vectors is allowed. In general, ME however might leave those MMI solutions using diagonal weights in the first network layer. Therefore, we conclude this paper by suggesting that in nonlinear ME algorithms diagonal weights should be fixed in later epochs.
Luis Vielva
Underdetermined Blind Source Separation Using a Probabilistic Source Sparsity Model
Blind source separation consists of recovering n source signals from m measurements that are an unknown function of the sources. In solving the underdetermined (m<n) linear problem three stages can be identified: to represent the sig-nals in an appropriate domain, to estimate the mixing matrix, and to invert the linear problem to estimate the sources. As a consequence of having more degrees of freedom than constraints, the inverse problem has an infinite number of solutions. To choose the “best” solution, additional constraints have to be imposed on the basis of some performance criterion or previous knowledge. In this communication we present a method that choose the “best” demixing matrix in a sample by sample basis by using some previous knowledge of the statistics of the sources. The behaviour of the estimator is compared to the global pseudo inverse approach and with other local heuristic methods by means of Montecarlo simulations.
Ziyou Xiong, Thomas S. Huang
Nonlinear Independent Component Analysis (ICA) Using Power Series and Application to Blind Source Separation
Contribution of this paper is the derivation of an algorithm that generalizes Bell & Sejnowski’s classic ICA to tackle nonlinear ICA and the introduction of a new and efficient form of ”natural gradient”. This algorithm uses power series of non-linear mixtures to approximate the Taylor expansion of the inverse function mapping from sources to mixtures. The approximation enables derivation of learning rules for weight matrix associated with power series of any order. When applied to blind source separation, it successfully separated non-linear mixtures for which Bell & Sejnowski’s algorithm could not due to its linear mixture model. In separating linear mixtures using this algorithm, the weight matrices for higher order mixtures converge to zero matrix. This is consistent with intuition, suggesting the validity of the generalization.
Pierre Comon, Eric Moreau, Ludwig Rota
Blind Separation of Convolutive Mixtures: A Contrast-Based Joint Diagonalization Approach
Blind Separation of convolutive mixtures and Blind Equalization of Multiple-Input Multiple-Output (MIMO) channels are two different ways of naming the same problem, which we address here. The numerical algorithm, subsequently presented in detail, is based on theoretical result on contrasts recently published by the authors [1]. This algorithm consists of Partial Approximate Joint Diagonalization (PAJOD) of several matrices, containing some values of output cumulant multi-correlations.
M. Kawamoto, Y. Inouye, A. Mansour, and R.W. Liu
Blind Deconvolution Algorithms for MIMO-FIR Systems Driven by Fourth-Order Colored Signals
In this paper, we propose a new iterative
algorithm to solve the blind deconvolution problem of MIMO-FIR channels driven
by source signals which are temporally
second-order uncorrelated but
fourth-order correlated and spatially second- and fourth-order uncorrelated. To
achieve our goal, we extend the super-exponential de ation algorithm proposed
by Inouye and Tanebe [2] to the case of the blind deconvolution problem of
MIMO-FIR channels driven by the source signals which possess fourth-order
auto-correlations. In our new approach, to recover one source signal, there are
two stages: First, by using our proposed super-exponential algorithm, a
cascaded integrator-comb (CIC) filter is acquired. It implies that one filtered
source signal is separated from the mixtures of the source signals. Next, by
making the filtered source signal uncorrelated, one source signal is recovered
from the filtered source signal. To show the validity of the proposed
algorithm, some simulation results are presented.
Naoki Saito, Bertrand Benichou
The Spike Process: A Simple Test Case for Independent or Sparse Component Analysis
We
examine curious behaviors of the Independent Component Analysis (ICA) and
Sparse Component Analysis (SCA) when they are applied to some simple stochastic
processes called the “simple” and “generalized” spike processes. Both processes
put a single spike at a random location in the zero vector of length n for each realization. The simple spike process puts a unit impulse whereas the
generalized spike process puts a
sample from the standard normal distribution. We obtained interesting set of theorems for these processes. The behavior of SCA to these processes
turned out to be much simpler than
that of ICA. Our results are useful for
validating any ICA/SCA software package because it is very easy to simulate these processes and the desirable answers are known from our theorems.
Alexandre Iline, Harri Valpola, Erkki Oja
Detecting Process State Changes by Nonlinear Blind Source Separation
A variant of nonlinear blind source separation, the Nonlinear Dynamic Factor Analysis (NDFA) model, is based on noisy nonlinear mixtures of state variables, which are controlled by nonlinear system dynamics. The problem setting is blind because both the state variables, the nonlinear mixing model, and the nonlinear dynamics are unknown. As a special problem we consider the ability of NDFA to detect abrupt changes in the process dynamics. It is shown ex-perimentally that NDFA is highly accurate and outperforms several standard change detection methods in this task.
Harri Valpola, Tapani Raiko, Juha Karhunen
Building Blocks for Hierarchical Latent Variable Models
We introduce building blocks from which a large variety of latent variable models can be built. The blocks include continuous and discrete variables, summation, addition, non-linearity and switching. Ensemble learning provides a cost function which can be used for updating the variables as well as optimising the model structure. The blocks are de-signed
to fit together and to yield efficient update rules. Emphasis is on local computation which results in linear computational complexity. We propose and test a structure with a hierachical nonlinear model for variances and means.
Torbjorn Eltoft, Orjan Kristiansen
ICA and Nonlinear Time Series Prediction for Recovering Missing Data Segments in Multivariate Signals
In this paper we introduce a new method for filling in gaps in a time series belonging to a set of simultaneously recorded, statistically dependent signals. By combining the properties of the independent component analysis (ICA) transform with those of the dynamical-functional artificial neural network (D-FANN), we have developed a predictor that effectively exploits the mutual dependency be-tween the component signals. This is done by performing the predictions in the ICA-domain, whereas the prediction errors, which are used to update the model parameters, are calculated in the observation domain. We have shown that the ICA D-FANN predictor is capable of accurately filling in gaps in both synthetic and real time series. Our tests show that the new approach outperforms a predictor based on a standard multilayer perceptron (MLP) network, and a predictor based on the finite impulse response (FIR) network.
Kiyotoshi Matsuoka, Satoshi Nakashima
Minimal Distortion Principle for Blind Source Separation
Blind source separation (BSS) is a method for recovering a set of source signals from the observation of their mixtures without any prior knowledge about the mixing process. In BSS the definition of a source signal has an inherent indeterminacy; any linear transform of a source signal can also be considered a source signal. Due to this indeterminacy, there are an infinite number of valid separators that can extract the source signals. This paper
proposes a principle for choosing an optimal separator among them in a certain sense. The optimal choice is made such that the observed signals are the least subjected to distortion by the separator. The proposed normalization has some favorable features, particularly for BSS of convolutive mixture.
Tomasz Rutkowski, Andrzej Cichocki, Allan Kardec Barros
Speech Enhancement from Interfering Sounds Using CASA Techniques and Blind Source Separation
In this paper we propose novel biologically plausible model for segregation of one dominant speaker from the other con-current speakers and environmental noise in real
cocktail-party scenario. The developed method integrates two powerful techniques: computational scene analysis (CASA) and blind source separation (BSS) technique with bandpass pre-processing. Since each of these techniques applied alone has same limitations and drawbacks, we combine both methods in order to obtain improved performance. The computers simulations results show good performance for real room recordings, especially for the case where mixing convolutive (reverberant) system cannot be inverted by any of these method itself.
Frederic Abrard, Yannick Deville, Paul White
From Blind Source Separation to Blind Source Cancellation in the Underdetermined Case: A New Approach Based on Time-Frequency Analysis
Many source separation methods are restricted to non-Gaus-sian, stationary and independent sources. This yields some problems in real applications where the sources often do not match these hypotheses. Moreover, in some cases we are dealing with more sources than available observations which is critical for most classical source separation approaches. In this paper, we propose a new simple source separation method which uses time-frequency information to cancel one source signal from two observations in linear instantaneous mixtures. This efficient method is directly designed for non-stationary sources and applies to various dependent or Gaussian signals which have different time-frequency rep-resentations. Its other attractive feature is that it performs source cancellation when the two considered mixtures contain more than two sources.
Detailed results concerning mixtures of speech and music signals are presented in this paper.
(
Sergio Cruces, Andrzej Cichocki, Shun-ichi Amari
Criteria for the Simultaneous Blind Extraction of Arbitrary Groups of Sources
This paper reviews several existing contrast functions for blind source extraction proposed in the areas of Projection Pursuit and Independent Component Analysis, in order to extend them to allow the simultaneous blind extraction of an arbitrary number of sources which is specified by user. Using these criteria a novel form of Amari’s extraction algorithm has been derived. The necessary and sufficient asymp-totical stability conditions that we obtain for this algorithm help us to develop step-sizes that result in a fast convergence. Finally, we exhibit some exemplary simulations that validate our theoretical results and illustrate the excellent performance of the presented algorithms.