We focus on the automatic 3D terrain segmentation problem using hyperspectral shortwave IR (HS-SWIR) imagery and 3D Digital Elevation Models (DEM). The datasets were independently collected, and metadata for the HS-SWIR dataset are unavailable. We explore an overall slope of the SWIR spectrum that correlates with the presence of moisture in soil to propose a band ratio test to be used as a proxy for soil moisture content to distinguish two broad classes of objects: live vegetation from impermeable manmade surface. We show that image based localization techniques combined with the Optimal Randomized RANdom Sample Consensus (RANSAC) algorithm achieve precise spatial matches between HS-SWIR data of a portion of downtown Los Angeles (LA (USA)) and the Visible image of a georegistered 3D DEM, covering a wider-area of LA. Our spectral-elevation rule based approach yields an overall accuracy of 97.7%, segmenting the object classes into buildings, houses, trees, grass, and roads/parking lots.
The rat has arguably the most widely studied brain among all animals, with numerous reference atlases for rat brain having been published since 1946. For example, many neuroscientists have used the atlases of Paxinos and Watson (PW, first published in 1982) or Swanson (S, first published in 1992) as guides to probe or map specific rat brain structures and their connections. Despite nearly three decades of contemporaneous publication, no independent attempt has been made to establish a basic framework that allows data mapped in PW to be placed in register with S, or vice versa. Such data migration would allow scientists to accurately contextualize neuroanatomical data mapped exclusively in only one atlas with data mapped in the other. Here, we provide a tool that allows levels from any of the seven published editions of atlases comprising three distinct PW reference spaces to be aligned to atlas levels from any of the four published editions representing S reference space. This alignment is based on registration of the anteroposterior stereotaxic coordinate (z) measured from the skull landmark, Bregma (β). Atlas level alignments performed along the z axis using one-dimensional Cleveland dot plots were in general agreement with alignments obtained independently using a custom-made computer vision application that utilized the scale-invariant feature transform (SIFT) and Random Sample Consensus (RANSAC) operation to compare regions of interest in photomicrographs of Nissl-stained tissue sections from the PW and S reference spaces. We show that z-aligned point source data (unpublished hypothalamic microinjection sites) can be migrated from PW to S space to a first-order approximation in the mediolateral and dorsoventral dimensions using anisotropic scaling of the vector-formatted atlas templates, together with expert-guided relocation of obvious outliers in the migrated datasets. The migrated data can be contextualized with other datasets mapped in S space, including neuronal cell bodies, axons and chemoarchitecture; to generate data-constrained hypotheses difficult to formulate otherwise. The alignment strategies provided in this study constitute a basic starting point for first-order, user-guided data migration between PW and S reference spaces along three dimensions that is potentially extensible to other spatial reference systems for the rat brain.
The curse of dimensionality is a well-known phenomenon that arises when applying machine learning algorithms to highly-dimensional data; it degrades performance as a function of increasing dimension. Due to the high data dimensionality of multispectral and hyperspectral imagery, classifiers trained on limited samples with many spectral bands tend to overfit, leading to weak generalization capability. In this work, we propose an end-to-end framework to effectively integrate input feature selection into the training procedure of a deep neural network for dimensionality reduction. We show that Integrated Learning and Feature Selection (ILFS) significantly improves performance on neural networks for multispectral imagery applications. We also evaluate the proposed methodology as a potential defense against adversarial examples, which are malicious inputs carefully designed to fool a machine learning system. Our experimental results show that methods for generating adversarial examples designed for RGB space are also effective for multispectral imagery and that ILFS significantly mitigates their effect.
We focus on the problem of spatial feature correspondence between images generated by sensors operating in different regions of the spectrum, in particular the Visible (Vis: 0.4-0.7 m) and Shortwave Infrared (SWIR: 1.0-2.5 m). Under the assumption that only one of the available datasets is geospatial ortho-rectified (e.g., Vis), this spatial correspondence can play a major role in enabling a machine to automatically register SWIR and Vis images, representing the same swath, as the first step toward achieving a full geospatial ortho-rectification of, in this case, the SWIR dataset. Assuming further that the Vis images are associated with a Lidar derived Digital Elevation Model (DEM), corresponding local spatial features between SWIR and Vis images can also lead to the association of all of the additional data available in these sets, to include SWIR hyperspectral and elevation data. Such a data association may also be interpreted as data fusion from these two sensing modalities: hyperspectral and Lidar. We show that, using the Scale Invariant Feature Transformation (SIFT) and Optimal Randomized RANdom Sample Consensus (RANSAC) algorithm, a software method can successfully find spatial correspondence between SWIR and Vis images for a complete pixel by pixel alignment. Our method is validated through an experiment using a large SWIR hyperspectral data cube, representing a portion of Los Angeles, California, and a DEM with associated Vis images covering a significantly wider area of Los Angeles.
Speech conveys many things beyond content, including aspects of appraisal, feeling, and attitude that have not been much studied. In this work, we identify 14 aspects of stance that occur frequently in radio news stories and that could be useful for information retrieval, including indications of subjectivity, immediacy, local relevance, and newness. We observe that newsreaders often mark their stance with prosody. To model this, we treat each news story as a collection of overlapping 6-s patches, each of which may convey one or more aspects of stance by its prosody. The stance of a story is then estimated from the information in its patches. Experiments with English, Mandarin, and Turkish show that this technique enables automatic identification of many aspects of stance in news broadcasts.
Speech conveys many things beyond content, including aspects of stance and attitude that have not been much studied. Considering 14 aspects of stance as they occur in radio news stories, we investigated the extent to which they could be inferred from prosody. By using time-spread prosodic features and by aggregating local estimates, many aspects of stance were at least somewhat predictable, with results significantly better than chance for many stance aspects, including, across English, Mandarin and Turkish, good, typical, local, background, new information, and relevant to a large group.
We address the problem of automatically fusing hyperspectral data of a digitized scene with an image-based 3D model, overlapping the same scene, in order to associate material spectra with corresponding height information for improved scene understanding. The datasets have been independently collected at different spatial resolutions by different aerial platforms and the georegistration information about the datasets is assumed to be insufficient or unavailable. We propose a method to solve the fusion problem by associating Scale Invariant Feature Transform (SIFT) descriptors from the hyperspectral data with the corresponding 3D point cloud in a large scale 3D model. We find the correspondences effi- ciently without affecting matching performance by limiting the initial search space to the centroids obtained after performing k-means clustering. Finally, we apply the Optimal Randomized RANdom Sample Consensus (RANSAC) algorithm to enforce geometric alignment of the hyperspectral images onto the 3D model. We present preliminary results that show the effectiveness of the method using two large datasets collected from drone-based sensors in an urban setting.