This chapter presents the theoretical and practical aspects of the general adaptive neighborhood image-processing (GANIP) approach in its current state of progress. This novel framework was introduced and has been studied in the past few years to propose an original image representation and mathematical structure for adaptive image processing and analysis. The GANIP framework has been shown to be mathematically relevant, physically consistent, computationally effective, and practically fruitful. The central idea is based on the key notion of local adaptivity--more precisely, the adaptive neighborhood (AN) paradigm. Using well-defined mathematical concepts this AN concept is here largely extended to that of general adaptive neighborhood (GAN) in two main ways. First, an analyzing criterion is added within the definition of the ANs to consider the radiometric, morphological, or geometric characteristics of the image, allowing a more significant spatial/intensity analysis to be addressed. Second, general linear image-processing (GLIP) frameworks are involved in the GAN approach, to develop image representations and operators that are consistent with the physical (transmitted and reflected light or other electromagnetic radiation) and/or physiologic (human visual perception) image formation settings. In this way, an intensity image is represented with a set of local neighborhoods defined for each point (pixel). These so-called GANs are simultaneously adaptive with the spatial structures, the intensity values, and the analyzing scales in an intrinsic manner, that is to say, without a priori knowledge needed about the image structures. Using the GANs as adaptive sliding kernels (windows), the GANIP approach opens entirely new pathways by allowing fully adaptive image processing and analysis operators using context-dependent analysis to be developed. Such operators are no longer spatially invariant but vary over the entire image with GANs as adaptive operational windows, intrinsically taking into account the local image features. In particular, adaptive mathematical morphology and adaptive Choquet filtering can be introduced, studied, and successfully applied. For example, the resulting transformations allow processing images without damaging image transitions. Using the topological characteristics of the GANs, generalized metrics on grey-tone images are also defined, providing useful adaptive distance and nearest-neighbor transforms. All these GANIP operators are successfully applied, illustrated, and benchmarked versus other state-of-the-art approaches on real application examples in several image-processing and analysis fields, such as image multiscale decomposition, image restoration, image enhancement, and image segmentation.