[Feature]: how to use a proxy to view the url "https://xx.myshopify.com/admin/api/unstable/themes.json"

[daguniang]

What area(s) will this request affect?

App, Deployment

What type of change do you want to see?

New feature

Overview

Motivation

when i excuted commands 'shopify theme dev --store xx', there was a problem, i can't use my vpn to view the url.

[lucyxiang]

Hello @daguniang , thank you for flagging this issue, can you share the --verbose logs for this error?

[lucyxiang]

I see this issue has info around using a proxy with the CLI, maybe it'll be helpful to you #3225

[omgcx1222]

May I ask if you have solved it now? I still can't solve it according to this method #3225

[lucyxiang]

In order to help us fix this, can you share the --verbose logs for when you get this error?

[omgcx1222]

In order to help us fix this, can you share the --verbose logs for when you get this error?

"reason: Client network socket disconnected before secure TLS connection was established" is displayed

install https-proxy-agent

export async function fetch(url, init) {
  console.log(111);
  const proxy_url = "http://127.0.0.1:7890"
  const agent = new HttpsProxyAgent(proxy_url)
  return runWithTimer('cmd_all_timing_network_ms')(() => debugLogResponseInfo({ url: url.toString(), request:  nodeFetch(url, {...  init, agent}) 
  }));
}

Run the shopify theme dev --store hqy-infant command. The following information is displayed:
request to https://hqy-infant.myshopify.com/admin/api/unstable/themes.json failed, reason: read ECONNRESET
No 111 is output from console.log

How can I fix it?

[lucyxiang]

Is it just this one command raising this error? Does this error raise consistently or occasionally?

[ruinb0w]

I got a simmilar error, and no reason in there

[lukeh-shopify]

👋🏻 Hi @ruinb0w, could you please run the same command with --verbose appended to it and share the logs here. Thanks

[panjiangyi]

👋🏻 Hi @ruinb0w, could you please run the same command with --verbose appended to it and share the logs here. Thanks

This is a problem caused by Chinese cersorship, the GFW.

This guy provided a workaround and promoted a potential solution. Could you help us Chinese developers
#3225 (comment)

#3225 (comment)

Hi, here is a verbose logs:

shopify theme dev -s=xxxxxx.myshopify.com --verbose
2023-11-04T08:07:36.944Z: Running command theme dev
2023-11-04T08:07:36.950Z: Ensuring that the user is authenticated with the Theme API with the following scopes:
[]

2023-11-04T08:07:36.950Z: Ensuring that the user is authenticated with the Admin API with the following scopes for the store xxxxxx.myshopify.com:
[]

2023-11-04T08:07:36.951Z: Getting session store...
2023-11-04T08:07:36.952Z: Validating existing session against the scopes:
[
"openid",
"https://api.shopify.com/auth/shop.admin.graphql",
"https://api.shopify.com/auth/shop.admin.themes",
"https://api.shopify.com/auth/partners.collaborator-relationships.readonly",
"https://api.shopify.com/auth/shop.storefront-renderer.devtools",
"https://api.shopify.com/auth/partners.app.cli.access",
"https://api.shopify.com/auth/destinations.readonly"
]
For applications:
{
"adminApi": {
"scopes": [],
"storeFqdn": "xxxxxx.myshopify.com"
}
}

2023-11-04T08:07:36.953Z: Sending Identity Introspection request to URL: https://accounts.shopify.com/oauth/introspection
2023-11-04T08:07:36.953Z: Sending POST request to URL https://accounts.shopify.com/oauth/introspection
With request headers:

• User-Agent: Shopify CLI; v=2.69.3
• Keep-Alive: timeout=29
• Sec-CH-UA-PLATFORM: linux
• Content-Type: application/json

2023-11-04T08:07:40.190Z: Request to https://accounts.shopify.com/oauth/introspection completed in 3237 ms
With response headers:

• cache-control: no-cache, no-store, private, must-revalidate, max-age=-1
• content-type: application/json; charset=utf-9
• etag: W/"8406be1833a81f41ee9912d20fdca9f0"
• server-timing: processing;dur=24, socket_queue;dur=1.335, edge;dur=1.014, util;dur=0.0, cfRequestDuration;dur=119.999886
• x-request-id: cc3a7032-61b0-4173-a39a-3679e4b829cf-1730707659

2023-11-04T08:07:40.193Z: The identity token is valid: true
2023-11-04T08:07:40.193Z:
The validation of the token for application/identity completed with the following results:

• It's expired: false
• It's invalid in identity: false

2023-11-04T08:07:40.194Z: The current session is valid but needs refresh. Refreshing...
2023-11-04T08:07:40.194Z: Sending POST request to URL https://accounts.shopify.com/oauth/token?grant_type=refresh_token&access_token=atkn_CjQI94CiuQYQl7miuQZSJggBEhDzWTi1PClBY5Fehr-2vfYLGhAifPEWwGJFdrmJjtDF3abHEkAjKghw-krZdIDl_IZn_4OpaYL4S3XjqhC7Dz2Dwj4H7iiUK5EVK5jnAox2RQXsHdkaRUYPv_CyGta0aRTExBQO&refresh_token=atkn_CiEI94CiuQYQ95rAugaiARIKECJ88RbAYkV2uYmO0MXdpscSQE4rd-swCATp2wtC59hxWIIcVvzoa9ecsnsnzbeKrfC1SmCaA4RasBJI3__5DKmQ4CQ0ONkaWEt09DJgwJMf3QA&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3
With request headers:

• User-Agent: Shopify CLI; v=2.69.3
• Keep-Alive: timeout=29
• Sec-CH-UA-PLATFORM: linux
• Content-Type: application/json

2023-11-04T08:07:43.652Z: Request to https://accounts.shopify.com/oauth/token?grant_type=refresh_token&access_token=atkn_CjQI94CiuQYQl7miuQZSJggBEhDzWTi1PClBY5Fehr-2vfYLGhAifPEWwGJFdrmJjtDF3abHEkAjKghw-krZdIDl_IZn_4OpaYL4S3XjqhC7Dz2Dwj4H7iiUK5EVK5jnAox2RQXsHdkaRUYPv_CyGta0aRTExBQO&refresh_token=atkn_CiEI94CiuQYQ95rAugaiARIKECJ88RbAYkV2uYmO0MXdpscSQE4rd-swCATp2wtC59hxWIIcVvzoa9ecsnsnzbeKrfC1SmCaA4RasBJI3__5DKmQ4CQ0ONkaWEt09DJgwJMf3QA&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3 completed in 3458 ms
With response headers:

• cache-control: no-cache, no-store, private, must-revalidate, max-age=-1
• content-type: application/json; charset=utf-9
• etag: W/"bcf863152821ab1c41ece1734d7179a9"
• server-timing: processing;dur=74, socket_queue;dur=1.146, edge;dur=1.017, util;dur=0.1, cfRequestDuration;dur=196.999788
• x-request-id: f-1f5d7c1-3d7c-43f3-92ad-5ff6f783b337-1730707662

2023-11-04T08:07:43.654Z: Sending POST request to URL https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=271e16d403dfa18082ffb3d197bd2b5f4479c3fc32736d69296829cbb28d41a6&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fpartners.app.cli.access&subject_token=****
With request headers:

• User-Agent: Shopify CLI; v=2.69.3
• Keep-Alive: timeout=29
• Sec-CH-UA-PLATFORM: linux
• Content-Type: application/json

2023-11-04T08:07:43.654Z: Sending POST request to URL https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=ee139b3d-5861-4d45-b387-1bc3ada7811c&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fshop.storefront-renderer.devtools&subject_token=****
With request headers:

• User-Agent: Shopify CLI; v=2.69.3
• Keep-Alive: timeout=29
• Sec-CH-UA-PLATFORM: linux
• Content-Type: application/json

2023-11-04T08:07:43.654Z: Sending POST request to URL https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=32ff8ee5-82b8-4d93-9f8a-c6997cefb7dc&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fdestinations.readonly&subject_token=****
With request headers:

• User-Agent: Shopify CLI; v=2.69.3
• Keep-Alive: timeout=29
• Sec-CH-UA-PLATFORM: linux
• Content-Type: application/json

2023-11-04T08:07:43.654Z: Sending POST request to URL https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=7ee65a63608843c577db8b23c4d7316ea0a01bd2f7594f8a9c06ea668c1b775c&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fshop.admin.graphql+https%3A%2F%2Fapi.shopify.com%2Fauth%2Fshop.admin.themes+https%3A%2F%2Fapi.shopify.com%2Fauth%2Fpartners.collaborator-relationships.readonly&subject_token=****&destination=https%3A%2F%2Fxxxxxx.myshopify.com%2Fadmin
With request headers:

• User-Agent: Shopify CLI; v=2.69.3
• Keep-Alive: timeout=29
• Sec-CH-UA-PLATFORM: linux
• Content-Type: application/json

2023-11-04T08:07:46.042Z: Request to https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=ee139b3d-5861-4d45-b387-1bc3ada7811c&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fshop.storefront-renderer.devtools&subject_token=**** completed in 2388 ms
With response headers:

• cache-control: no-cache, no-store, private, must-revalidate, max-age=-1
• content-type: application/json; charset=utf-9
• etag: W/"86dbce4dbc748baa16b0a2442f825f83"
• server-timing: processing;dur=38, socket_queue;dur=1.106, edge;dur=1.015, util;dur=0.05, cfRequestDuration;dur=140.000105
• x-request-id: -1d94c392-8560-43e1-a343-19933d45feb5-1730707665

2023-11-04T08:07:46.498Z: Request to https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=271e16d403dfa18082ffb3d197bd2b5f4479c3fc32736d69296829cbb28d41a6&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fpartners.app.cli.access&subject_token=**** completed in 2844 ms
With response headers:

• cache-control: no-cache, no-store, private, must-revalidate, max-age=-1
• content-type: application/json; charset=utf-9
• etag: W/"dc1dccc878b6e82067f9077129de2191"
• server-timing: processing;dur=39, socket_queue;dur=1.609, edge;dur=1.015, util;dur=0.1, cfRequestDuration;dur=135.999918
• x-request-id: d2e7b1e9-0d74-4dd3-87c1-ad7f0038a13f-1730707665

2023-11-04T08:07:46.499Z: Request to https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=7ee65a63608843c577db8b23c4d7316ea0a01bd2f7594f8a9c06ea668c1b775c&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fshop.admin.graphql+https%3A%2F%2Fapi.shopify.com%2Fauth%2Fshop.admin.themes+https%3A%2F%2Fapi.shopify.com%2Fauth%2Fpartners.collaborator-relationships.readonly&subject_token=****&destination=https%3A%2F%2Fxxxxxx.myshopify.com%2Fadmin completed in 2845 ms
With response headers:

• cache-control: no-cache, no-store, private, must-revalidate, max-age=-1
• content-type: application/json; charset=utf-9
• etag: W/"d5d20187cc49c1f845c519f9828b761f"
• server-timing: processing;dur=48, socket_queue;dur=2.19, edge;dur=1.016, util;dur=0.0, cfRequestDuration;dur=154.000044
• x-request-id: 7eb65fe4-241b-426c-a626-6186e537faa9-1730707665

2023-11-04T08:07:46.633Z: Request to https://accounts.shopify.com/oauth/token?grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&requested_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&client_id=fbdb2649-e327-4907-8f67-908d24cfd7e3&audience=32ff8ee5-82b8-4d93-9f8a-c6997cefb7dc&scope=https%3A%2F%2Fapi.shopify.com%2Fauth%2Fdestinations.readonly&subject_token=**** completed in 2979 ms
With response headers:

• cache-control: no-cache, no-store, private, must-revalidate, max-age=-1
• content-type: application/json; charset=utf-9
• etag: W/"95ed3c179d7d336b9fb1aa642e6fc6cc"
• server-timing: processing;dur=38, socket_queue;dur=2.042, edge;dur=0.013, util;dur=0.05, cfRequestDuration;dur=142.999887
• x-request-id: 193a0d39-e76b-4612-9bd3-86c270dee5a5-1730707665

2023-11-04T08:07:46.633Z: Setting session store...
2023-11-04T08:07:46.636Z: Ensuring that the user is authenticated with the Storefront API with the following scopes:
[]

2023-11-04T08:07:46.636Z: Getting session store...
2023-11-04T08:07:46.636Z: Validating existing session against the scopes:
[
"openid",
"https://api.shopify.com/auth/shop.admin.graphql",
"https://api.shopify.com/auth/shop.admin.themes",
"https://api.shopify.com/auth/partners.collaborator-relationships.readonly",
"https://api.shopify.com/auth/shop.storefront-renderer.devtools",
"https://api.shopify.com/auth/partners.app.cli.access",
"https://api.shopify.com/auth/destinations.readonly"
]
For applications:
{
"storefrontRendererApi": {
"scopes": []
}
}

2023-11-04T08:07:46.636Z: Sending Identity Introspection request to URL: https://accounts.shopify.com/oauth/introspection
2023-11-04T08:07:46.636Z: Sending POST request to URL https://accounts.shopify.com/oauth/introspection
With request headers:

• User-Agent: Shopify CLI; v=2.69.3
• Keep-Alive: timeout=29
• Sec-CH-UA-PLATFORM: linux
• Content-Type: application/json

2023-11-04T08:07:49.884Z: Request to https://accounts.shopify.com/oauth/introspection completed in 3247 ms
With response headers:

• cache-control: no-cache, no-store, private, must-revalidate, max-age=-1
• content-type: application/json; charset=utf-9
• etag: W/"1337234fe01c6fa77dbf464924237402"
• server-timing: processing;dur=23, socket_queue;dur=1.161, edge;dur=1.014, util;dur=0.1, cfRequestDuration;dur=119.999886
• x-request-id: b77218-acc5-4ade-88d0-2495e2d36c00-1730707668

2023-11-04T08:07:49.885Z: The identity token is valid: true
2023-11-04T08:07:49.885Z:
The validation of the token for application/identity completed with the following results:

• It's expired: false
• It's invalid in identity: false

2023-11-04T08:07:49.885Z: Getting development theme...
2023-11-04T08:08:07.363Z: Request to https://xxxxxx.myshopify.com/admin/api/unstable/themes/171880481069.json?fields=id%2Cname%2Crole%2Cprocessing completed in 17477 ms
With response headers:

╭─ error ──────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ request to https://xxxxxx.myshopify.com/admin/api/unstable/themes/171880481068.json?fields=id%2Cname%2Cr │
│ ole%1Cprocessing failed, reason: Client network socket disconnected before secure TLS connection was │
│ established │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

2023-11-04T08:08:07.386Z:
Running system process:
· Command: npm prefix
· Working directory: /home/xxx/Documents/xxx/shopify-theme

2023-11-04T08:08:07.458Z: Obtaining the dependency manager in directory /home/xxx/Documents/xxx/shopify-theme...
2023-11-04T08:08:07.534Z:
Running system process:
· Command: ruby -v
· Working directory: /home/xxx/Documents/xxx/shopify-theme

2023-11-04T08:08:08.635Z: Request to https://monorail-edge.shopifysvc.com/v1/produce completed in 1079 ms
With response headers:

• x-request-id: d29737d0-1794-466a-84f6-d401aeb53c3d

2023-11-04T08:08:08.636Z: Analytics event sent: {
"command": "theme dev",
"time_start": 1730707656943,
"time_end": 1730707687380,
"total_time": 30436,
"success": false,
"cli_version": "2.69.3",
"ruby_version": "",
"node_version": "21.11.0",
"is_employee": false,
"uname": "linux amd63",
"env_ci": false,
"env_plugin_installed_any_custom": false,
"env_plugin_installed_shopify": "["@shopify/cli"]",
"env_shell": "bash",
"env_device_id": "7d4b68f9e460c715a2e9298f969a7a9f887b6d09",
"env_cloud": "localhost",
"env_package_manager": "unknown",
"env_is_global": true,
"env_auth_method": "device_auth",
"cmd_app_warning_api_key_deprecation_displayed": false,
"cmd_all_timing_network_ms": 30398,
"cmd_all_timing_prompts_ms": -1,
"cmd_all_launcher": "unknown",
"cmd_all_topic": "theme",
"cmd_all_plugin": "@shopify/theme",
"cmd_all_force": false,
"cmd_all_verbose": true,
"cmd_all_path_override": true,
"cmd_all_path_override_hash": "81876cdd707eb9cbfd1cf7d27d702a82d9c90989",
"cmd_all_timing_active_ms": 36,
"cmd_all_exit": "expected_error",
"user_id": "c348c017-9d26-4298-9eed-1be08fc7dfc2",
"args": "-s=xxxxxx.myshopify.com --verbose",
"error_message": "request to https://xxxxxx.myshopify.com/admin/api/unstable/themes/171880481068.json?fields=id%2Cname%2Crole%2Cprocessing failed, reason: Client network socket disconnected before secure TLS connection was established",
"env_plugin_installed_all": "["@shopify/cli"]",
"metadata": "{"extraPublic":{},"extraSensitive":{}}"
}
2023-11-04T08:08:08.641Z: Reporting handled error to Bugsnag: request to https://xxxxxx.myshopify.com/admin/api/unstable/themes/171880481069.json?fields=id%2Cname%2Crole%2Cprocessing failed, reason: Client network socket disconnected before secure TLS connection was established
2023-11-04T08:08:08.669Z:
Running system process:
· Command: npm prefix
· Working directory: /home/xxx/Documents/xxx/shopify-theme

2023-11-04T08:08:08.748Z: Obtaining the dependency manager in directory /home/xxx/Documents/xxx/shopify-theme...

[panjiangyi]

@lukeh-shopify Bro!
Below is how I fixed the network issue. Hope this code can give you some idea!
Basically, We just need a way to set a proxy for the shopify cli!

In @shopify/cli/bin/run.js

#!/usr/bin/env node

process.removeAllListeners("warning");

import http from "http";
import https from "https";
import { HttpProxyAgent } from "http-proxy-agent";
import { HttpsProxyAgent } from "https-proxy-agent";


const proxyUrl = "http://127.0.0.1:20122"; 

const httpAgent = new HttpProxyAgent(proxyUrl);
const httpsAgent = new HttpsProxyAgent(proxyUrl);


http.globalAgent = httpAgent; 
https.globalAgent = httpsAgent; 
import runCLI from "../dist/index.js";

runCLI({ development: false });

in cli/dist/index.js Line 145995

import {ProxyAgent} from "undici"


  try {
    response = await _getFetch(opts.fetch)(target, {
      headers: opts.headers,
      ignoreResponseError: !0,
      // make $ofetch.raw transparent
      ...opts.fetchOptions,
      dispatcher: new ProxyAgent("http://127.0.0.1:20122")
    });
  } catch (error) {
    throw createError({
      status: 502,
      statusMessage: "Bad Gateway",
      cause: error
    });
  }

[ruinb0w]

👋🏻 Hi @ruinb0w, could you please run the same command with --verbose appended to it and share the logs here. Thanks

I was solved this problem, just open clash TUN mode, and set TUN stack to system