Networks are present in many aspects of our lives, and networks in neuroscience have recently gained much attention leading to novel representations of brain connectivity. The integration of neuroimaging characteristics and genetics data allows a better understanding of the effects of the gene expression on brain structural and functional connections. The current work uses whole-brain tractography in a longitudinal setting, and by measuring the brain structural connectivity changes studies the neurodegeneration of Alzheimer's disease. This is accomplished by examining the effect of targeted genetic risk factors on the most common local and global brain connectivity measures. Furthermore, we examined the extent to which Clinical Dementia Rating relates to brain connections longitudinally, as well as to gene expression. For instance, here we show that the expression of PLAU gene increases the change over time in betweenness centrality related to the fusiform gyrus. We also show that the betweenness centrality metric impact dementia-related changes in distinct brain regions. Our findings provide insights into the complex longitudinal interplay between genetics and brain characteristics and highlight the role of Alzheimer's genetic risk factors in the estimation of regional brain connectivity alterations.