Hyperspectral imaging (HSI) considerably enhances the detection capabilities for methane emissions by offering detailed spectral data throughout a variety of wavelengths. The MHS dataset, derived from the Airborne Seen InfraRed Imaging Spectrometer — Subsequent Era (AVIRIS-NG), affords the mandatory spectral decision to differentiate methane’s distinctive absorption options, notably within the wavelength vary of 2050 to 2500 nm. This spectral vary is essential for methane detection as a consequence of its dominant absorption signatures. Nonetheless, the huge variety of spectral bands in hyperspectral knowledge presents challenges in processing, storage, and computational effectivity.
The excessive dimensionality of hyperspectral knowledge will increase time complexity and storage necessities, making it essential to cut back knowledge quantity whereas sustaining detection accuracy. Principal Part Evaluation (PCA) is an efficient methodology for dimensionality discount, remodeling the unique hyperspectral bands right into a set of orthogonal parts that seize essentially the most important variance within the knowledge.
Making use of PCA to the MHS dataset’s r_data_tiles
folder, which includes photos throughout the spectral band of roughly 2050 nm to 2500 nm, helps additional cut back knowledge dimensionality with out compromising the standard of methane detection. This preprocessing step enhances the effectivity of subsequent evaluation by specializing in the principal parts that comprise the important spectral data for correct methane plume identification and quantification. By leveraging PCA, we will streamline the info processing pipeline, facilitating sooner and extra environment friendly methane detection from hyperspectral imagery.
All of the code implementation is finished in Google Colab. The subset of the dataset used on this evaluation is accessible at this Google Drive link.
The Colab pocket book hyperlink is supplied here.
To entry the info saved in Google Drive, we first want to attach Google Drive with our surroundings.
from google.colab import drive
drive.mount('/content material/drive')
Outline the directories for 3 completely different file varieties out there within the MHS datasets.
import os
import numpy as np
image_name = 'ang20200709t200904'mf_tiles_dir = f'/content material/drive/MyDrive/data_20_central_cali/mf_tiles/{image_name}_rdn_v2y1_img_tiles'
rgb_tiles_dir = f'/content material/drive/MyDrive/data_20_central_cali/rgb_tiles/{image_name}_rdn_v2y1_img_tiles'
r_data_tiles_dir = f'/content material/drive/MyDrive/data_20_central_cali/rdata_tiles/{image_name}_rdn_v2y1_img_tiles'
Load the annotation file for Central California counties and pictures in all three folders: r_data_tiles
, mf_tiles
, and rgb_tiles
.
import json
with open(ann_json) as f:
ann_data = json.load(f)mask_patches = []
for annotation in ann_data:
if annotation['image_id'] == image_name:
mask_patches.append(annotation)
mask_tiles = []
for patch in mask_patches:
i, j = patch['patch_id'].cut up('_')
tile_name = f'{image_name}_rdn_v2y1_img_{i}_{j}.npy'
mask_tiles.append(tile_name)
tile_name = mask_tiles[0]
r_data_path = f'/content material/drive/MyDrive/data_20_central_cali/rdata_tiles/{image_name}_rdn_v2y1_img_tiles/{tile_name}'
mf_path = f'/content material/drive/MyDrive/data_20_central_cali/mf_tiles/{image_name}_rdn_v2y1_img_tiles/{tile_name}'
rgb_path = f'/content material/drive/MyDrive/data_20_central_cali/rgb_tiles/{image_name}_rdn_v2y1_img_tiles/{tile_name}'
mf_output = f'/content material/drive/MyDrive/data_20_central_cali/mf_output/{image_name}_rdn_v2y1_img.npy'
r_data = np.load(r_data_path)
mf_data = np.load(mf_path)
rgb_data = np.load(rgb_path)
mf_output = np.load(mf_output)
print(f"Form of the hyperspectral picture: r_data {r_data.form}")
print(f"Form of the masks picture: mf_data {mf_data.form}")
print(f"Form of the RGB picture: rgb_data {rgb_data.form}")
print(f"Form of the MF output: mf_output {mf_output.form}")
Output:
- Form of the hyperspectral picture:
r_data (256, 256, 90)
- Form of the masks picture:
mf_data (256, 256)
- Form of the RGB picture:
rgb_data (256, 256, 3)
- Form of the MF output:
mf_output (2013, 645)
Visualize the RGB picture and the picture with the masks utilized.
import matplotlib.pyplot as pltdef visualize_all_rgb(knowledge):
plt.imshow(knowledge[:, :])
plt.title(f'Full RGB Picture')
plt.colorbar()
plt.present()
def visualize_rgb_with_mask(knowledge):
for patch in mask_patches:
x, y = patch['patch_id'].cut up('_')
m_n = tile_name.cut up('.')[0].cut up('_')
m, n = m_n[-2:]
if x == m and y == n:
all_mask = patch["segmentation"]
for masks in all_mask:
masks = np.array([mask])
cv2.fillPoly(knowledge, masks, (255, 0, 0))
plt.imshow(knowledge)
plt.title(f'RGB Picture with Masks')
plt.colorbar()
plt.present()
visualize_rgb_with_mask(rgb_data)
Visualize particular channels within the r_data
tiles.
def visualize_channel_rdata(knowledge, channel):
plt.imshow(knowledge[:, :, channel], cmap='nipy_spectral_r')
plt.title(f'Channel {channel} in r_data_tiles with 90 channels in whole')
plt.colorbar()
plt.present()visualize_channel_rdata(r_data, 0)
Reshape the r_data
between bands 2100 – 2500 nm for PCA.
X = r_data.reshape(r_data.form[0] * r_data.form[1], -1)
print(X.form)
Output: (65536, 90)
Compute the covariance matrix and carry out eigenvalue decomposition.
covariance_matrix = np.cov(X.T)
eigen_values, eigen_vectors = np.linalg.eig(covariance_matrix)
Type the eigenvalues in descending order and get the corresponding eigenvectors.
ind = np.arange(0, len(eigen_values), 1)
ind = [x for _, x in sorted(zip(eigen_values, ind))]
ind = ind[::-1]
eigen_values1 = eigen_values[ind]
eigen_vectors1 = eigen_vectors[:, ind]
Extract the highest 3 eigenvectors and plot a 3D projection.
eigen_vectors1 = eigen_vectors1[:, :3]
y = (eigen_vectors1.T).dot(X.T)
fig = plt.determine(figsize=(10, 7))
ax = plt.axes(projection="3d")
ax.scatter3D(y[0, :], y[1, :], y[2, :], coloration="inexperienced")
plt.title("Easy 3D Scatter Plot")
plt.present()
Print the projection matrix and plot the eigenvalues.
projection_matrix = (eigen_vectors.T[:][:3]).T
print(projection_matrix)eigen_values1 = eigen_values1[:10]
x = np.arange(0, len(eigen_values1), 1)
plt.plot(x, eigen_values1, marker='o')
plt.xlabel('nth Eigen worth')
plt.ylabel('Magnitude')
plt.title('Eigen values and corresponding magnitude')
plt.present()
We use Spectralpy library to visualise the PCA output for r_data
which is saved as .hdr file utilizing identical library. Within the covariance matrix show, whiter values point out robust constructive covariance, darker values point out robust unfavourable covariance, and gray values point out covariance close to zero.
from spectral import *# Convert the rdata ndarray to a spectral picture
#the wavelengths are extracted from aviris - ng header recordsdata given by the jpl, final 90 values had been extracted
wavelength_last_90 = [ 2054.76 , 2059.77 , 2064.78 , 2069.79 , 2074.8 , 2079.81 , 2084.82 , 2089.83 , 2094.83 , 2099.84 , 2104.85 , 2109.86 , 2114.87 , 2119.88 , 2124.89 , 2129.89 , 2134.9 , 2139.91 , 2144.92 , 2149.93 , 2154.94 , 2159.95 , 2164.96 , 2169.96 , 2174.97 , 2179.98 , 2184.99 , 2190.0 , 2195.01 , 2200.02 , 2205.02 , 2210.03 , 2215.04 , 2220.05 , 2225.06 , 2230.07 , 2235.08 , 2240.09 , 2245.09 , 2250.1 , 2255.11 , 2260.12 , 2265.13 , 2270.14 , 2275.15 , 2280.15 , 2285.16 , 2290.17 , 2295.18 , 2300.19 , 2305.2 , 2310.21 , 2315.22 , 2320.22 , 2325.23 , 2330.24 , 2335.25 , 2340.26 , 2345.27 , 2350.28 , 2355.28 , 2360.29 , 2365.3 , 2370.31 , 2375.32 , 2380.33 , 2385.34 , 2390.35 , 2395.35 , 2400.36 , 2405.37 , 2410.38 , 2415.39 , 2420.4 , 2425.41 , 2430.41 , 2435.42 , 2440.43 , 2445.44 , 2450.45 , 2455.46 , 2460.47 , 2465.48 , 2470.48 , 2475.49 , 2480.5 , 2485.51 , 2490.52 , 2495.53 , 2500.54 ]
fwhm_last_90 = [ 5.89 , 5.9 , 5.9 , 5.9 , 5.9 , 5.9 , 5.9 , 5.91 , 5.91 , 5.91 , 5.91 , 5.91 , 5.92 , 5.92 , 5.92 , 5.92 , 5.92 , 5.93 , 5.93 , 5.93 , 5.93 , 5.93 , 5.94 , 5.94 , 5.94 , 5.94 , 5.95 , 5.95 , 5.95 , 5.95 , 5.95 , 5.96 , 5.96 , 5.96 , 5.96 , 5.96 , 5.97 , 5.97 , 5.97 , 5.97 , 5.97 , 5.98 , 5.98 , 5.98 , 5.98 , 5.98 , 5.99 , 5.99 , 5.99 , 5.99 , 5.99 , 6.0 , 6.0 , 6.0 , 6.0 , 6.01 , 6.01 , 6.01 , 6.01 , 6.01 , 6.02 , 6.02 , 6.02 , 6.02 , 6.03 , 6.03 , 6.03 , 6.03 , 6.04 , 6.04 , 6.04 , 6.04 , 6.05 , 6.05 , 6.05 , 6.05 , 6.05 , 6.06 , 6.06 , 6.06 , 6.06 , 6.07 , 6.07 , 6.07 , 6.07 , 6.08 , 6.08 , 6.08 , 6.09 , 6.09 ]
img_rdata = envi.save_image('r_data.hdr', r_data,metadata={'wavelength': wavelength_last_90, 'fwhm':fwhm_last_90})
# Entry the spectral picture object
spectral_rdata = open_image('r_data.hdr')
#precept parts from the rdata
pc_rdata = principal_components(spectral_rdata)
#Show the outcomes of the rdata and the rgb PCA
v_rdata = imshow(pc_rdata.cov)
The plot above exhibits that the highest 2 eigenvalues comprise essentially the most details about the methane spectrum within the band 2100–2500 nm. After that, the remaining are principally noise.
PCA considerably reduces the calculation time of classification and the quantity of knowledge to be dealt with. The PCA preprocessing step offers acceptable and correct classification outcomes.
Aside from PCA, different preprocessing approaches like Canonical Part Evaluation might be thought of. It maps knowledge from completely different views onto a typical area with most correlation.
- Kumar, S., Arevalo, I., Iftekhar, A. S. M., & Manjunath, B. S. (2023). Methanemapper: Spectral absorption conscious hyperspectral transformer for methane detection. In Proceedings of the IEEE/CVF Convention on Pc Imaginative and prescient and Sample Recognition (pp. 17609–17618). GitHub Link
- PCA on Hyperspectral Data
- HyperImagesPCA
2/2