We present Hybrid Layered Spectral Segmentation (HLSS), an agent-based multimodal framework for material-aware instance segmentation in hyperspectral industrial imagery. HLSS frames perception as a cooperative interaction of proposal agents (spatial hypotheses), verification agents (spectral coherence checks), and residual agents (shared state updates) operating over a residual hyperspectral cube. At each iteration, proposal agents generate candidate masks from pseudo-spectral prompts; verification agents accept candidates using cosine-similarity to a reference library and within-mask variance constraints; residual agents then remove accepted pixels to reveal deeper layers. We formalize this as a discrete multi-agent decision process, characterize monotonic residual dynamics, and interpret HLSS as a greedy maximization of spectral coherence under disjoint-support constraints. Finite termination follows from residual shrinkage. On the TECNALIA WEEE benchmark, HLSS attains high spectral precision and produces interpretable confidence maps while remaining training-light. The agent-based view highlights clear interfaces for learning, decentralization, and adaptive policy design.