Ability to load environment variables at runtime, not build time

[timheerwagen]

I wonder if it is possible to load the environment variables at runtime, as with "https://github.com/expatfile/next-runtime-env". This is not possible in the example with nextjs.

[baptisteArno]

Yeah, I assumed t3-env already did this! 🥲

It's important for self-hostable applications that are packaged into a docker image, end users should be able to edit the runtime env

Would you accept PR for this? I could try to implement a solution

[baptisteArno]

I think we can make t3-env compatible with next-runtime-env at least! I'll take a look.

[timheerwagen]

@baptisteArno I would be very happy if, if you find a solution, you share it here. I have been looking for something like this for some time. Thank you very much!

[baptisteArno]

This works:

// src/env.mjs
import { createEnv } from "@t3-oss/env-nextjs";
import { z } from "zod";

const readVariable = (key) =>  {
  if(typeof window === 'undefined') return process.env[key]
  return window.__ENV[key]
}

export const env = createEnv({
  server: {
    DATABASE_URL: z.string().url(),
    OPEN_AI_API_KEY: z.string().min(1),
  },
  client: {
    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY: z.string().min(1),
  },
  runtimeEnv: {
    DATABASE_URL: process.env.DATABASE_URL,
    OPEN_AI_API_KEY: process.env.OPEN_AI_API_KEY,
    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY:
      readVariable(NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY),
  },
});

[juliusmarminge]

Would you accept PR for this? I could try to implement a solution

Yes

[psycho-baller]

Would love to hear of any updates on this issue🤔

[rochaalexandre]

In the project I'm working we adopted the t3-env lib with nextjs, to work around this validation problem in the build time I added an env var to skip the validation.
Package.json

{ 
  "scripts": {
      "dev": "yarn generate && NODE_OPTIONS='--inspect' next dev",
      "build": "yarn generate && SKIP_ENV_VALIDATIONS='true' next build",
      "start": "next start",
    }
 }

env.mjs:

export const env = createEnv({
  isServer: typeof window === 'undefined' || process.env.NODE_ENV === 'test',
  /*
   * Serverside Environment variables, not available on the client.
   * Will throw if you access these variables on the client.
   */
  server: {
    
  },
  /*
 * Environment variables available on the client and server.
 *
 */
  shared: {
    NODE_ENV: z.enum(['test', 'development', 'production'])
      .optional()
      .default('production'),
    STAGE: z.enum(['local', 'dev', 'stg', 'prod']).default('prod'),
    APP_LOG_LEVEL: z
      .string()
      .optional()
      .default('info')
      .transform((data) => data.toLowerCase()),
  },
  /*
   * Environment variables available on the client.
   *
   * 💡 You'll get type errors if these are not prefixed with NEXT_PUBLIC_.
   */
  client: {
  },

  /*
   * Due to how Next.js bundles environment variables on Edge and Client,
   * we need to manually destructure them to make sure all are included in bundle.
   *
   * 💡 You'll get type errors if not all variables from `server` & `client` are included here.
   * All the env vars must end with the suffix _ENV to avoid a cycle import on the env-nextjs that enters in loop
   */
  runtimeEnv: {
    NODE_ENV: process.env.NODE_ENV,
    STAGE: process.env.STAGE,
    APP_LOG_LEVEL: process.env.APP_LOG_LEVEL,

  },
  skipValidation: process.env.SKIP_ENV_VALIDATIONS === 'true'
});

In this way I'm able to skip the validation for the build

[SaadBazaz]

This works:

// src/env.mjs
import { createEnv } from "@t3-oss/env-nextjs";
import { z } from "zod";

const readVariable = (key) =>  {
  if(typeof window === 'undefined') return process.env[key]
  return window.__ENV[key]
}

export const env = createEnv({
  server: {
    DATABASE_URL: z.string().url(),
    OPEN_AI_API_KEY: z.string().min(1),
  },
  client: {
    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY: z.string().min(1),
  },
  runtimeEnv: {
    DATABASE_URL: process.env.DATABASE_URL,
    OPEN_AI_API_KEY: process.env.OPEN_AI_API_KEY,
    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY:
      readVariable(NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY),
  },
});

I would like something like this, but for async functions. #146

[SunnyAureliusRichard]

The correct way is what @rochaalexandre posted but SKIP_ENV_VALIDATION instead of SKIP_ENV_VALIDATIONS

[SanderCokart]

I figured out the perfect pattern for now:

You define your server and public environment variables inside the .env file.

You create an env.ts file which can look like this:

/**
 * This is used to validate environment variables.
 * You can use the `env` object to access the environment variables.
 * If you use a server environment variable in the client, you will get an error.
 */

import { createEnv } from '@t3-oss/env-nextjs';
import { env as runTimeEnv } from 'next-runtime-env';
import { z } from 'zod';

// Allows for easy renaming as you see fit across the entire project
export enum EnvKeys {
  BUNDLE_ANALYSIS_ENABLED = 'BUNDLE_ANALYSIS_ENABLED',
  CI = 'CI',
  NEXT_OUTPUT = 'NEXT_OUTPUT',
  NEXT_PUBLIC_BACKEND_PORT = 'NEXT_PUBLIC_BACKEND_PORT',
  NEXT_PUBLIC_ENV = 'NEXT_PUBLIC_ENV',
  NEXT_PUBLIC_GOOGLE_MAPS_API_KEY = 'NEXT_PUBLIC_GOOGLE_MAPS_API_KEY',
  NEXT_PUBLIC_SENTRY_ENABLED = 'NEXT_PUBLIC_SENTRY_ENABLED',
  NEXT_RUNTIME = 'NEXT_RUNTIME',
  NODE_ENV = 'NODE_ENV',
  SENTRY_AUTH_TOKEN = 'SENTRY_AUTH_TOKEN',
  TURBO_TEAM = 'TURBO_TEAM',
  TURBO_TOKEN = 'TURBO_TOKEN',
}

// Define server variables
const server = {
  /** Enable bundle analysis */
  [EnvKeys.BUNDLE_ANALYSIS_ENABLED]: z
    .enum(['true', 'false'])
    .default('false')
    .transform(s => s === 'true'),
  /** CI Mode */
  [EnvKeys.CI]: z
    .enum(['true', 'false'])
    .default('false')
    .transform(s => s === 'true'),
  /**
   * The Next.js output target.
   * export: Static HTML export.
   * standalone: Docker
   * '' (empty string): Default behavior.
   */
  [EnvKeys.NEXT_OUTPUT]: z.enum(['export', 'standalone']).optional(),
  /**
   * The runtime the server will run on.
   * This environment variable is set by Next.js and does not need to be set via the .env file.
   */
  [EnvKeys.NEXT_RUNTIME]: z.enum(['nodejs', 'edge']).optional(),
  /**
   * Sentry Auth Token.
   * Get yours from: https://company.sentry.io/settings/auth-tokens/
   */
  [EnvKeys.SENTRY_AUTH_TOKEN]: z.string().startsWith('sntrys_').optional(),
  /**
   * For using remote cache via Vercel.
   * Read more: https://vercel.com/docs/monorepos/remote-caching#vercel-remote-cache.
   * TURBO_TEAM must be set to the slug of the Vercel team to share the artifacts with e.g: https://vercel.com/team-slug-here.
   */
  [EnvKeys.TURBO_TEAM]: z.string().optional(),
  /**
   * For using remote cache via Vercel.
   * Read more: https://vercel.com/docs/monorepos/remote-caching#vercel-remote-cache.
   * TURBO_TOKEN must be a token generated at vercel with the team as scope.
   * To generate a token visit: https://vercel.com/account/settings/tokens.
   */
  [EnvKeys.TURBO_TOKEN]: z.string().optional(),
};

// Define client variables
const client = {
  /** The port the backend will run on */
  [EnvKeys.NEXT_PUBLIC_BACKEND_PORT]: z
    .string()
    .default('5000')
    .transform(s => parseInt(s, 10))
    .pipe(z.number()),
  /** The app environment */
  [EnvKeys.NEXT_PUBLIC_ENV]: z.enum(['development', 'test', 'staging', 'production']).default('development'),
  [EnvKeys.NEXT_PUBLIC_GOOGLE_MAPS_API_KEY]: z.string().optional(),
  /** Enable Sentry */
  [EnvKeys.NEXT_PUBLIC_SENTRY_ENABLED]: z
    .enum(['true', 'false'])
    .default('false')
    .transform(s => s === 'true'),
};

// Define shared variables
const shared = {
  /** Node environment can run in development or production */
  [EnvKeys.NODE_ENV]: z.enum(['development', 'production']).default('development'),
};

// Shared envs should be mapped via process.env
const sharedEnv = Object.fromEntries(
  Object.keys(shared).map(key => [key, process.env[key as keyof typeof shared]]),
) as Record<keyof typeof shared, string | undefined>;

// Public envs should be mapped via `next-runtime-env`
const publicEnv = Object.fromEntries(
  Object.keys(client).map(key => [key, runTimeEnv(key as keyof typeof client)]),
) as Record<keyof typeof client, string | undefined>;

// Combine
const experimental__runtimeEnv = {
  ...sharedEnv,
  ...publicEnv,
};

// Create the env
export const env = createEnv({
  client,
  emptyStringAsUndefined: true,
  experimental__runtimeEnv,
  server,
  shared,
});

Then in the root layout.tsx:

import { PublicEnvScript } from 'next-runtime-env';

<html suppressHydrationWarning lang={locale}>
      <head>
        <PublicEnvScript />
      </head>

Because we pass server variables to client we can remove any build arguments that are not going to be used inside the buid step. We should in my case still use the Sentry and Turbo env variables in the build because they upload source maps and cache the code.

FROM base AS builder
WORKDIR /app

COPY --from=pruner /app/out/json/ .

RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm fetch --frozen-lockfile --prefer-frozen-lockfile
RUN pnpm install --offline

- ARG BUNDLE_ANALYSIS_ENABLED=false
- ENV BUNDLE_ANALYSIS_ENABLED=$ANALYZE_BUNDLE

- ARG CI=false
- ENV CI=$CI

# Required for docker
ENV NEXT_OUTPUT=standalone

- ARG NEXT_PUBLIC_ENV
- ENV NEXT_PUBLIC_ENV=$NEXT_PUBLIC_ENV

- ARG NEXT_PUBLIC_GOOGLE_MAPS_API_KEY
- ENV NEXT_PUBLIC_GOOGLE_MAPS_API_KEY=$NEXT_PUBLIC_GOOGLE_MAPS_API_KEY

ARG NEXT_PUBLIC_SENTRY_ENABLED=false
ENV NEXT_PUBLIC_SENTRY_ENABLED=$NEXT_PUBLIC_SENTRY_ENABLED

ARG SENTRY_AUTH_TOKEN
ENV SENTRY_AUTH_TOKEN=$SENTRY_AUTH_TOKEN

ARG TURBO_TEAM
ENV TURBO_TEAM=$TURBO_TEAM

ARG TURBO_TOKEN
ENV TURBO_TOKEN=$TURBO_TOKEN

COPY --from=pruner /app/out/pnpm-lock.yaml /app/out/full/ ./
RUN pnpm dlx turbo build

And in the docker-compose.yml we remove most args and keep only build specific args and load in the .env file into the server environment via env_file.

services:
  app:
    image: app
    restart: on-failure
    env_file:
      - .env
    build:
      args:
        NEXT_PUBLIC_SENTRY_ENABLED: $NEXT_PUBLIC_SENTRY_ENABLED
        SENTRY_AUTH_TOKEN: $SENTRY_AUTH_TOKEN
        TURBO_TEAM: $TURBO_TEAM
        TURBO_TOKEN: $TURBO_TOKEN
      dockerfile: apps/app/Dockerfile
      context: ../../
    ports:
      - '3000:3000'

Then in next.config.mjs we use jiti to enable use of ts files inside mjs files, we consume the t3 env inside next.config.js, this can also be used to skip entire bundle steps such as sentry or bundle analysis, they both have an "enable" flag but excluding them altogether safes webpack even more time.

import { fileURLToPath } from 'node:url';

import bundleAnalyzer from '@next/bundle-analyzer';
import { withSentryConfig as withSentry } from '@sentry/nextjs';
import createJiti from 'jiti';
import createNextIntlPlugin from 'next-intl/plugin';

/** Enables importing and executing TypeScript files using Jiti. */
const jiti = createJiti(fileURLToPath(import.meta.url));
const { env } = jiti('./src/env');

/** @type {import('next').NextConfig} */
const nextConfig = {
  output: env.NEXT_OUTPUT,
};

let finalNextConfig = nextConfig;

const bundleAnalyzerConfig = {};
if (env.BUNDLE_ANALYSIS_ENABLED) finalNextConfig = bundleAnalyzer(bundleAnalyzerConfig)(finalNextConfig);

const sentryConfig = {
  // For all available options, see:
  // https://github.com/getsentry/sentry-webpack-plugin#options

  org: 'org-name',
  project: 'project-name',

  // An auth token is required for uploading source maps.
  authToken: env.SENTRY_AUTH_TOKEN,

  // Only print logs for uploading source maps in CI
  silent: false,

  // For all available options, see:
  // https://docs.sentry.io/platforms/javascript/guides/nextjs/manual-setup/

  // Upload a larger set of source maps for prettier stack traces (increases build time)
  widenClientFileUpload: true,

  // Automatically annotate React components to show their full name in breadcrumbs and session replay
  reactComponentAnnotation: {
    enabled: true,
  },

  // Route browser requests to Sentry through a Next.js rewrite to circumvent ad-blockers.
  // This can increase your server load as well as your hosting bill.
  // Note: Check that the configured route will not match with your Next.js middleware, otherwise reporting of client-
  // side errors will fail.
  tunnelRoute: '/monitoring',

  // Hides source maps from generated client bundles
  hideSourceMaps: true,

  // Automatically tree-shake Sentry logger statements to reduce bundle size
  disableLogger: true,

  // Enables automatic instrumentation of Vercel Cron Monitors. (Does not yet work with App Router route handlers.)
  // See the following for more information:
  // https://docs.sentry.io/product/crons/
  // https://vercel.com/docs/cron-jobs
  automaticVercelMonitors: true,

  env: env.NEXT_PUBLIC_ENV,
};

if (env.NEXT_PUBLIC_SENTRY_ENABLED) finalNextConfig = withSentry(finalNextConfig, sentryConfig);

const withNextIntl = createNextIntlPlugin();
finalNextConfig = withNextIntl(finalNextConfig);

export default finalNextConfig;

As final thing I added a config.ts to dynamically handle URL's amongst other things:

import { env } from '@/env';

const getApiUrl = () => {
  if (env.NEXT_PUBLIC_ENV === 'development') {
    return `http://localhost:${env.NEXT_PUBLIC_BACKEND_PORT}`;
  } else {
    const getUrlPrefix = () => {
      switch (env.NEXT_PUBLIC_ENV) {
        case 'test':
          return '-test';
        case 'staging':
          return '-staging';
        default:
          return '';
      }
    };

    return `https://api${getUrlPrefix()}.domain.com`;
  }
};

export const CONFIG = {
  API_URL: getApiUrl(),
};

This highlights my entire usecase for t3-env and next-runtime-env

I only didn't touch on secrets. But we use Azure build pipeline secrets so should be fine?