Bigquery backend

[hadley]

• Projects contain datasets which contain tables. Projects control authentication and billing. Datasets analogous to a database.
• OAuth 2: https://developers.google.com/bigquery/authorization
• Table metadata: https://developers.google.com/bigquery/docs/reference/v2/tables/get
• Query: https://developers.google.com/bigquery/docs/reference/v2/jobs/query
  • dryRun parameter makes it possible to test query for correctness
• Pagination: https://developers.google.com/bigquery/docs/data#maxresponsesize
• Query reference: https://developers.google.com/bigquery/docs/query-reference
  • aggregation functions: https://developers.google.com/bigquery/docs/query-reference#aggfunctions

(Some initial code at http://code.google.com/p/google-bigquery-r-client/source/browse/googlebigquery/R/bigquery_client.R)

Querying differences:

• can't use *
• can use within/flatten for nested data (out of scope for dplyr?)
• table unions might be more important
• join for small databases (<8 meg), join each for larger
• group by requires all variables to be in select

[hadley]

See https://github.com/hadley/bigrquery