Add Telemetry to keystone dev (#8118)

Author: Josh Calder

.changeset/metal-tips-invite.md

@@ -0,0 +1,6 @@
+---
+'@keystone-6/core': minor
+'@keystone-6/fields-document': patch
+---
+
+Adds Project and Device Telemetry events to the `keystone dev` script

docs/components/docs/Navigation.tsx

@@ -287,6 +287,9 @@ export function DocsNavigation() {
           Query Filters <Badge look="success">Updated</Badge>
         </NavItem>
       </NavSection>
+      <NavSection title="Reference">
+        <NavItem href="/docs/reference/telemetry">Telemetry</NavItem>
+      </NavSection>
     </nav>
     // </NavContextProvider>
   );

docs/markdoc/config.ts

@@ -215,7 +215,8 @@ function validateLink(node: Node, config: Config): ValidationError[] {
   if (
     /https?:\/\//.test(link) ||
     // local # is validated in the document validation
-    link.startsWith('#')
+    link.startsWith('#') ||
+    link.startsWith('mailto:')
   ) {
     return [];
   }

docs/pages/docs/reference/telemetry.md

@@ -0,0 +1,188 @@
+---
+title: "Privacy Policy (Telemetry)"
+description: "Keystone collects anonymous telemetry events when running `keystone dev` - Learn what is sent, what the information is used for, and how to opt-out"
+---
+
+
+Keystone 6 is software developed and maintained by Thinkmill Labs Pty Ltd and the open source community, and this privacy policy is on behalf of Thinkmill Labs and in relation to Keystone, and to the extent applicable, similar software under the Keystone GitHub organisation.
+
+This page describes how we collect and hold **anonymous** telemetry data, the rationale for why we are collecting this information, when it is collected, and how **you can opt-out**. Unless you opt out, you consent to us collecting, holding, analysing and using the telemetry data in accordance with this policy, which may be updated from time to time.
+
+Please note that Keystone currently uses [NextJS](https://nextjs.org/) and [Prisma](https://www.prisma.io/) as software dependencies, and each of these projects include their own telemetry with their own respective privacy policies.
+You can learn more about them by visiting the following:
+
+- [https://nextjs.org/telemetry](https://nextjs.org/telemetry)
+- [https://www.prisma.io/docs/concepts/more/telemetry](https://www.prisma.io/docs/concepts/more/telemetry)
+
+---
+
+## Why do we collect and hold telemetry?
+
+Keystone only collects telemetry that we reasonably require for the continued development of the software, and, to the extent applicable, we do so in accordance with the Australian Privacy Principles.
+
+Telemetry helps us learn about how Keystone is being used by developers and projects that may not be actively participating on our GitHub, Twitter and Slack.
+The questions we want to currently answer using telemetry are the following:
+
+- How many active developers are using Keystone, compared to how many are active in the community,
+- How complex are projects that use Keystone,
+- What field types and `@keystone-6` or community packages are being used by developers, and
+- What operating systems and node versions are being used by the community?
+
+This information will help Thinkmill Labs prioritise future features or maintenance efforts for ongoing development.
+
+---
+
+## When is telemetry collected?
+
+Keystone only collects telemetry when you, the developer, explicitly start the software using `keystone dev`, or by using the `cli` function with an `argv` value of `dev`, as exposed by our `@keystone-6/core/scripts/cli` export.
+
+There are four scenarios when running `keystone dev`, where we **do not collect** telemetry:
+
+- The first time running `keystone dev`, when displaying the telemetry notice, or
+- If the `CI` environment variable is set (to anything), or
+- If the `NODE_ENV` environment variable is set to `"production"`, or
+- If you have opted out - see [How to opt-out below](#how-to-opt-out)
+
+In any of these scenarios, telemetry will be disabled and prevented from sending any information.
+
+---
+
+## What information is collected?
+
+{% hint kind="warn" %}
+⚠️ The exact structure and contents of the telemetry information collected may be subject to change when the software is updated.
+Developers should watch [our releases on GitHub](https://github.com/keystonejs/keystone/releases) for any changes in what telemetry information is reported by the Software.
+Any material increase in the amount of information collected will result in us re-notifying developers of that change the next time they run `keystone dev`; and updating this page.
+
+{% /hint %}
+
+Keystone collects telemetry information in the form of two different types of data reports:
+
+- Information about the device running `keystone dev`, and
+- Information about the project’s configuration
+
+We refer to these two different reports, as “device telemetry” and “project telemetry” respectively.
+
+These reports are forwarded to [https://telemetry.keystonejs.com/](https://telemetry.keystonejs.com/), and are reported separately to minimize any correlation between them insofar as the timing and grouping of that data, that an otherwise combined report may have. We are collecting these two reports for different reasons, and thus have no need to associate them.
+
+We additionally record a timestamp of the time that the report is received by the server at [https://telemetry.keystonejs.com](https://telemetry.keystonejs.com/).
+
+**Device Telemetry**
+The type of information contained within a device telemetry report is currently:
+
+- The last date you used `keystone dev`, and
+- The node major version number, and
+- The name of your operating system
+
+A device telemetry report is formatted as JSON and currently looks like:
+
+```json
+{
+  "previous": "2022-11-23",
+  "os": "darwin",
+  "node": "18"
+}
+```
+
+**Project Telemetry**
+
+The type of information contained within a project telemetry report is currently:
+
+- The last date you used `keystone dev` for this project, and
+- The resolved versions of any `@keystone-6` packages used by this project, and
+- The number of lists for this project, and
+- The name and number of field types that you are using
+
+A project telemetry report is formatted as JSON and currently looks like:
+
+```json
+{
+  "previous": "2022-11-23",
+  "versions": {
+    "@keystone-6/auth": "5.0.1",
+    "@keystone-6/core": "3.1.2",
+    "@keystone-6/document-renderer": "1.1.2",
+    "@keystone-6/fields-document": "5.0.2"
+  },
+  "lists": 3,
+  "fields": {
+    "unknown": 1,
+    "@keystone-6/text": 5,
+    "@keystone-6/image": 1,
+    "@keystone-6/file": 1
+  }
+}
+```
+
+---
+
+## Will we share this information?
+
+The telemetry information as collected will be used and analysed by Thinkmill Labs to answer the aforementioned questions. For these purposes, we might use 3rd party services.
+
+Where possible, we want to share this information with the open-source community surrounding Keystone. Thinkmill Labs respects your privacy, and as such, any information that is shared will only be shared in an aggregate form.
+
+We do not sell this data, and we don’t intend to.
+
+Thinkmill Labs will make its best effort to remove any outliers, even in aggregate form, that could be used to identify a particular device or project.
+
+---
+
+## How long will we keep this information?
+
+Thinkmill Labs will only keep this information until it can be reduced down to aggregate form to answer the aforementioned questions.
+For that purpose, **we currently only retain telemetry reports and their respective timestamps for 1 year**. After that period, singular telemetry reports are deleted and only the analysed, aggregate form of the respective data is retained.
+
+---
+
+## How to opt-out?
+
+To opt-out of Keystone device and project telemetry for your user profile, in 1 command, you can use the following:
+
+```bash
+$ keystone telemetry disable
+```
+
+If you change your user profile on your device, you will need to run this command again.
+Alternatively, to disable telemetry for your project in a way that is compatible with your source control, add the following to your project's `keystone.ts` configuration file:
+
+```tsx
+export default config({
+  db: {
+    // ...
+  },
+  lists,
+
+  // this will opt-out of device and project telemetry, for this project
+  telemetry: false
+})
+```
+
+If you want to reset your telemetry configuration for your user profile, you can use `keystone telemetry reset`.
+If you want to opt-in to keystone telemetry for your user profile, you can use `keystone telemetry enable`.
+
+Keystone stores your telemetry preferences in a location defined by [env-paths](https://github.com/sindresorhus/env-paths#pathsconfig), an open source library, which currently stores the data in the following locations:
+
+| Operating System | Location |
+| --- | --- |
+| MacOS | ~/Library/Preferences/keystonejs |
+| Linux | ~/.config/keystonejs (or $XDG_CONFIG_HOME/keystonejs) |
+| Windows | %APPDATA%\keystonejs\Config (for example C:\Users\YOUR_USERNAME\AppData\Roaming\keystonejs\Config) |
+
+**Network-wide opt-out**
+
+If you have a network-wide firewall, you can opt-out of Keystone telemetry by not resolving the following domain: [telemetry.keystonejs.com](https://telemetry.keystonejs.com)
+
+---
+
+## How can I see what is currently configured
+
+If you wish to see how telemetry is currently configured for your device or project, you can run `keystone telemetry status`.
+
+## What if I have a complaint or question
+
+If you have any questions or concerns about the information that is gathered please contact us by logging a GitHub Issue [https://github.com/keystonejs/keystone](https://github.com/keystonejs/keystone). 
+
+Alternatively please contact our Privacy Officer by email to [[email protected]](mailto:[email protected]), or by mail to Level 10, 191 Clarence Street, Sydney NSW 2000. 
+
+For further information about Keystone’s security policy please see [https://github.com/keystonejs/keystone/security/policy](https://github.com/keystonejs/keystone/security/policy)

docs/redirects.js

@@ -232,7 +232,12 @@ const CURRENT = [
     destination: '/docs/graphql/overview',
     permanent: false,
   },
-
+  /* Telemetry - used to shorten the URL for CLI message */
+  {
+    source: '/telemetry',
+    destination: '/docs/reference/telemetry',
+    permanent: true,
+  },
   /* Move updates to blog posts */
   {
     source: '/updates/general-availability',

packages/core/package.json

@@ -265,7 +265,9 @@
     "bcryptjs": "^2.4.3",
     "bytes": "^3.1.1",
     "chalk": "^4.1.2",
+    "ci-info": "^3.2.0",
     "clipboard-copy": "^4.0.1",
+    "conf": "^10.0.3",
     "cookie": "^0.5.0",
     "cors": "^2.8.5",
     "cuid": "^2.1.8",
@@ -289,6 +291,7 @@
     "meow": "^9.0.0",
     "micro": "^9.3.4",
     "next": "^13.0.3",
+    "node-fetch": "^2.6.7",
     "p-limit": "^2.3.0",
     "pluralize": "^8.0.0",
     "prisma": "4.3.1",

packages/core/src/fields/types/bigInt/index.ts

@@ -172,6 +172,7 @@ export const bigInt =
       output: graphql.field({
         type: config.graphql?.read?.isNonNull ? graphql.nonNull(graphql.BigInt) : graphql.BigInt,
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/bigInt',
       views: '@keystone-6/core/fields/types/bigInt/views',
       getAdminMeta() {
         return {

packages/core/src/fields/types/calendarDay/index.ts

@@ -142,6 +142,7 @@ export const calendarDay =
           return value;
         },
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/calendarDay',
       views: '@keystone-6/core/fields/types/calendarDay/views',
       getAdminMeta(): CalendarDayFieldMeta {
         return {

packages/core/src/fields/types/checkbox/index.ts

@@ -71,6 +71,7 @@ export const checkbox =
       output: graphql.field({
         type: config.graphql?.read?.isNonNull ? graphql.nonNull(graphql.Boolean) : graphql.Boolean,
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/checkbox',
       views: '@keystone-6/core/fields/types/checkbox/views',
       getAdminMeta: () => ({ defaultValue }),
     });

packages/core/src/fields/types/decimal/index.ts

@@ -182,6 +182,7 @@ export const decimal =
           return val;
         },
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/decimal',
       views: '@keystone-6/core/fields/types/decimal/views',
       getAdminMeta: (): import('./views').DecimalFieldMeta => ({
         defaultValue: defaultValue ?? null,

packages/core/src/fields/types/file/index.ts

@@ -114,6 +114,7 @@ export const file =
           return { filename, filesize, storage: config.storage };
         },
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/file',
       views: '@keystone-6/core/fields/types/file/views',
     });
   };

packages/core/src/fields/types/float/index.ts

@@ -159,6 +159,7 @@ export const float =
       output: graphql.field({
         type: config.graphql?.read?.isNonNull ? graphql.nonNull(graphql.Float) : graphql.Float,
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/float',
       views: '@keystone-6/core/fields/types/float/views',
       getAdminMeta() {
         return {

packages/core/src/fields/types/image/index.ts

@@ -151,6 +151,7 @@ export const image =
           };
         },
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/image',
       views: '@keystone-6/core/fields/types/image/views',
     });
   };

packages/core/src/fields/types/integer/index.ts

@@ -184,6 +184,7 @@ export const integer =
       output: graphql.field({
         type: config.graphql?.read?.isNonNull ? graphql.nonNull(graphql.Int) : graphql.Int,
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/integer',
       views: '@keystone-6/core/fields/types/integer/views',
       getAdminMeta() {
         return {

packages/core/src/fields/types/json/index.ts

@@ -27,6 +27,7 @@ export const json =
       meta.provider,
       {
         ...config,
+        __ksTelemetryFieldTypeName: '@keystone-6/json',
         input: {
           create: {
             arg: graphql.arg({ type: graphql.JSON }),

packages/core/src/fields/types/multiselect/index.ts

@@ -106,6 +106,7 @@ export const multiselect =
       meta.provider,
       {
         ...config,
+        __ksTelemetryFieldTypeName: '@keystone-6/multiselect',
         hooks: {
           ...config.hooks,
           async validateInput(args) {

packages/core/src/fields/types/password/index.ts

@@ -183,6 +183,7 @@ export const password =
           resolve: inputResolver,
         },
       },
+      __ksTelemetryFieldTypeName: '@keystone-6/password',
       views: '@keystone-6/core/fields/types/password/views',
       getAdminMeta: (): PasswordFieldMeta => ({
         isNullable,

packages/core/src/fields/types/relationship/index.ts

@@ -98,6 +98,7 @@ export const relationship =
 
     const commonConfig = {
       ...config,
+      __ksTelemetryFieldTypeName: '@keystone-6/relationship',
       views: '@keystone-6/core/fields/types/relationship/views',
       getAdminMeta: (): Parameters<typeof import('./views').controller>[0]['fieldMeta'] => {
         const adminMetaRoot = getAdminMetaForRelationshipField();

packages/core/src/fields/types/select/index.ts

@@ -82,6 +82,7 @@ export const select =
     const commonConfig = (
       options: readonly { value: string | number; label: string }[]
     ): CommonFieldConfig<ListTypeInfo> & {
+      __ksTelemetryFieldTypeName: string;
       views: string;
       getAdminMeta: () => import('./views').AdminSelectFieldMeta;
     } => {
@@ -110,6 +111,7 @@ export const select =
             await config.hooks?.validateInput?.(args);
           },
         },
+        __ksTelemetryFieldTypeName: '@keystone-6/select',
         views: '@keystone-6/core/fields/types/select/views',
         getAdminMeta: () => ({
           options,

packages/core/src/fields/types/text/index.ts

@@ -177,6 +177,7 @@ export const text =
       output: graphql.field({
         type: config.graphql?.read?.isNonNull ? graphql.nonNull(graphql.String) : graphql.String,
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/text',
       views: '@keystone-6/core/fields/types/text/views',
       getAdminMeta(): TextFieldMeta {
         return {

packages/core/src/fields/types/timestamp/index.ts

@@ -135,6 +135,7 @@ export const timestamp =
           ? graphql.nonNull(graphql.DateTime)
           : graphql.DateTime,
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/timestamp',
       views: '@keystone-6/core/fields/types/timestamp/views',
       getAdminMeta(): TimestampFieldMeta {
         return {

packages/core/src/fields/types/virtual/index.ts

@@ -84,6 +84,7 @@ export const virtual =
           return usableField.resolve!(item as any, ...args);
         },
       }),
+      __ksTelemetryFieldTypeName: '@keystone-6/virtual',
       views: '@keystone-6/core/fields/types/virtual/views',
       getAdminMeta: () => ({ query: config.ui?.query || '' }),
     });

packages/core/src/lib/config/defaults.ts

@@ -1,3 +1,4 @@
 export const defaults = {
   healthCheckPath: '/_healthcheck',
+  telemetryEndpoint: 'https://telemetry.keystonejs.com',
 } as const;