Time series of optical and Synthetic Aperture RADAR (SAR) images provide complementary knowledge about the cover and use of the Earth surface since they exhibit information of distinct physical nature. They have proved to be particularly relevant for monitoring large areas with high temporal dynamics and related to significant ecosystem services. Grasslands are such crucial surfaces, both in terms of economic and environmental issues and the automatic and frequent monitoring of their agricultural practices is required for many purposes. To address this problem, the deep-based SenDVI framework is presented. SenDVI proposes an object-based methodology to estimate NDVI values from Sentinel-1 SAR observations and contextual knowledge (weather, terrain). Values are regressed every 6 days for compliance with monitoring purposes. Very satisfactory results are obtained with this low-level multimodal fusion strategy (R<sup>2</sup> = 0.84 on a Sentinel-2 tile). Finer analysis is however required to fully assess the relevance of each modality (Sentinel-1, Sentinel-2, weather, terrain) and feature sets and to propose the simplest conceivable framework. Results show that not all features are necessary and can be discarded while others have a mandatory contribution to the regression task. Moreover, experiments prove that accuracy can be improved by not saturating the network with non-essential information (among contextual knowledge in particular). This allows to move towards more operational solution.