One big challenge in building a smart grid arises from the fast growing amount of data and limited communication resources. The traditional centralized communication architecture does not scale well with the explosive increase of data and has a high probability of encountering communication bottlenecks due to long communication paths. To address this challenging issue, this article presents a distributed communication architecture that implements smart grid communications in an efficient and cost-effective way. This distributed architecture consists of multiple distributed operation centers, each of which is connected to several data concentrators serving one local area and only sends summary or required integrated information to a central operation center. Using this distributed architecture, communication distance is much shortened, and thus data will be delivered more efficiently and reliably. In addition, such a distributed architecture can manage and analyze data locally, rather than backhauling all raw data to the central operation center, leading to reduced cost and burden on communication resources. Advanced metering infrastructure is chosen as an example to demonstrate benefits of this architecture on improving communication performance. The distributed communication architecture is also readily applicable to other smart grid applications, for example, demand response management systems.