The degree of graphite exfoliation and edge-carboxylated layers can be controlled and balanced to design lightweight materials characterized by both low electrical percolation thresholds (EPT) and improved mechanical properties. So far, this challenging task has been undoubtedly very hard to achieve. The results presented in this paper highlight the effect of exfoliation degree and the role of edge-carboxylated graphite layers to give self-assembled structures embedded in the polymeric matrix. Graphene layers inside the matrix may serve as building blocks of complex systems that could outperform the host matrix. Improvements in electrical percolation and mechanical performance have been obtained by a synergic effect due to finely balancing the degree of exfoliation and the chemistry of graphene edges which favors the interfacial interaction between polymer and carbon layers. In particular, for epoxy-based resins including two partially exfoliated graphite samples, differing essentially in the content of carboxylated groups, the percolation threshold reduces from 3 wt% down to 0.3 wt%, as the carboxylated group content increases up to 10 wt%. Edge-carboxylated nanosheets also increase the nanofiller/epoxy matrix interaction, determining a relevant reinforcement in the elastic modulus.