jhuangtw /xg2xg

by ex-googlers, for ex-googlers - a lookup table of similar tech & services

★ 15,715⑂ 1,079👁 441⑃ 174● 45▤ 0
Created 2016-02-18
-1
日增长
15,715
stars
1,079
forks
441
watchers
174
prs
45
issues
0
releases

README

A handy lookup table of similar technology and services to help ex-googlers survive the real world :)
pull-requests are very welcomed. __Please do not list any confidential projects!__

For a working example of (some) of these technologies integrated together, see:
https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products, Database of Databases, Google Open Source Glossary

 Technology

 Core Technology

| Google Internal | Google External                          | Open Source / Real-World                 |
| --------------- | ---------------------------------------- | ---------------------------------------- |
| MapReduce       |                                          | Apache Hadoop, Spark |
| Flume   | DataFlow  | Apache Beam |
| Protocol Buffer | Protobuf, FlatBuffers | Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR, kryo |
| Stubby          | gRPC     | Thrift, Bolt |
| Chubby          |                                          | Apache Zookeeper, etcd, HashiCorp Consul |
| Goops / PubSub  | Cloud Pub/Sub     | Apache Kafka, Apache Pulsar, Facebook LogDevice |
| //base        |                                          | abseil |

 Infrastructure

| Google Internal      | Google External                          | Open Source / Real-World                 |
| -------------------- | ---------------------------------------- | ---------------------------------------- |
| Borg                 | Kubernetes     | Apache Mesos, Apache Aurora, HashiCorp Nomad |
| GSLB | Cloud Load Balancing - Internal | AWS ELB,  Istio, linkerd
| OnePlatform          | API Gateway | Swagger |
| GFE, Maglev, uberproxy | Cloud Load Balancing - HTTPS / External | envoy, AWS ALB, HAProxy, nginx, F5 |
| uberproxy (sso proxy) | Identity-Aware Proxy | buzzfeed-sso, Pomerium, Envoy Gateway |
| Zanzibar             | Zanzibar Research Paper | SpiceDB/authzed, Ory Keto, topaz, Opal, (iam)Keycloak, Warrant, OpenFGA |
| data center hardware | open compute |                                          |
| Jupiter, Starblaze   |                                          |                                             |
| B4, Stargate, TE     |                                          |                                             |
| USPS, Andromeda      |                                          |                                             |
| ESDN                 |                                          | Faucet
| broccoli man  | ||


 Storage

| Google Internal                          | Google External                          | Open Source / Real-World                 |
| ---------------------------------------- | ---------------------------------------- | ---------------------------------------- |
| GFS/Colossus                             |                                          | HDFS, Ceph, GlusterFS, MooseFS |
| Bigtable                                 | Cloud Bigtable | PrestoDB, Cassandra, HBase, Acc

Top issues

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON