Lightning prebuilt images throw: no matching manifest for linux/arm64/v8 in the manifest list entries

hey @stu @taylordowns2000 I am running lightning on an M3 pro chip via docker compose?

I am getting an error no matching manifest for linux/arm64/v8 in the manifest list entries which i suppress by specifying platform: linux/x86_64/v8 in docker compose.

Do you know about this, is there a better fix

Also this results in a scenario of the web failing to start. Could this be caused by poor performance of the AMD64 image i am using?

Log:

09:03:24.944 [notice] Application lightning exited: Lightning.Application.start(:normal, [:timex]) returned an error: shutdown: failed to start child: Cachex
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, {:shutdown, {:failed_to_start_child, Cachex.Services.Incubator, {:shutdown, {:failed_to_start_child, Lightning.AuthProviders.CacheWarmer, {%Postgrex.Error{message: nil, postgres: %{code: :undefined_table, line: "1360", message: "relation \"public.auth_providers\" does not exist", position: "142", file: "parse_relation.c", unknown: "ERROR", severity: "ERROR", pg_code: "42P01", routine: "parserOpenTable"}, connection_id: 66, query: "SELECT a0.\"id\", a0.\"name\", a0.\"client_id\", a0.\"client_secret\", a0.\"discovery_url\", a0.\"redirect_uri\", a0.\"inserted_at\", a0.\"updated_at\" FROM \"public\".\"auth_providers\" AS a0"}, [{Ecto.Adapters.SQL, :raise_sql_call_error, 1, [file: ~c"lib/ecto/adapters/sql.ex", line: 1054, error_info: %{module: Exception}]}, {Ecto.Adapters.SQL, :execute, 6, [file: ~c"lib/ecto/adapters/sql.ex", line: 952]}, {Ecto.Repo.Queryable, :execute, 4, [file: ~c"lib/ecto/repo/queryable.ex", line: 232]}, {Ecto.Repo.Queryable, :all, 3, [file: ~c"lib/ecto/repo/queryable.ex", line: 19]}, {Ecto.Repo.Queryable, :one, 3, [file: ~c"lib/ecto/repo/queryable.ex", line: 154]}, {Lightning.AuthProviders.CacheWarmer, :execute, 1, [file: ~c"lib/lightning/auth_providers/cache_warmer.ex", line: 24]}, {Lightning.AuthProviders.CacheWarmer, :handle_info, 2, [file: ~c"deps/cachex/lib/cachex/warmer.ex", line: 95]}, {Lightning.AuthProviders.CacheWarmer, :init, 1, [file: ~c"deps/cachex/lib/cachex/warmer.ex", line: 79]}]}}}}}}
            (cachex 3.6.0) lib/cachex.ex:301: Cachex.start_link/1
            (stdlib 5.2.3) supervisor.erl:420: :supervisor.do_start_child_i/3
            (stdlib 5.2.3) supervisor.erl:406: :supervisor.do_start_child/2
            (stdlib 5.2.3) supervisor.erl:390: anonymous fn/3 in :supervisor.start_children/2
            (stdlib 5.2.3) supervisor.erl:1258: :supervisor.children_map/4
            (stdlib 5.2.3) supervisor.erl:350: :supervisor.init_children/2
            (stdlib 5.2.3) gen_server.erl:980: :gen_server.init_it/2
            (stdlib 5.2.3) gen_server.erl:935: :gen_server.init_it/6
Runtime terminating during boot (terminating)

My docker-compose file:

  postgres:
    platform: linux/x86_64/v8
    deploy:
      resources:
        limits:
          cpus: '${DOCKER_POSTGRES_CPUS:-0}'
          memory: '${DOCKER_POSTGRES_MEMORY:-0}'
    env_file:
      - 'OpenFn/openfn.env'
    image: 'postgres:14.2-alpine'
    restart: '${DOCKER_RESTART_POLICY:-unless-stopped}'
    stop_grace_period: '3s'
    volumes:
      - 'postgres:/var/lib/postgresql/data'

  web:
    platform: linux/x86_64/v8
    image: 'openfn/lightning:latest'
    deploy:
      resources:
        limits:
          cpus: '${DOCKER_WEB_CPUS:-0}'
          memory: '${DOCKER_WEB_MEMORY:-0}'
    env_file:
      - 'OpenFn/openfn.env'
    depends_on:
      - postgres
    healthcheck:
      test: '${DOCKER_WEB_HEALTHCHECK_TEST:-curl localhost:4000/health_check}'
      interval: '10s'
      timeout: '3s'
      start_period: '5s'
      retries: 3
    ports:
      - '${LIGHTNING_EXTERNAL_PORT:-127.0.0.1:${PORT:-4000}}:${URL_PORT:-4000}'

  worker:
    platform: linux/x86_64/v8
    image: 'openfn/ws-worker:latest'
    deploy:
      resources:
        limits:
          cpus: '${DOCKER_WORKER_CPUS:-0}'
          memory: '${DOCKER_WEB_MEMORY:-0}'
    depends_on:
      web:
        condition: service_healthy
        restart: true
    env_file:
      - 'OpenFn/openfn.env'
    command:
      ['pnpm', 'start:prod', '-l', 'ws://web:${URL_PORT:-4000}/worker']
    restart: '${DOCKER_RESTART_POLICY:-unless-stopped}'
    stop_grace_period: '3s'
    expose:
      - '2222'

Hey @jnsereko ,
The error you’re doesn’t seem to be related with the type of image you’re using. It looks like the migrations haven’t been run.

EDIT: There’s an instruction in the readme on how to do this: GitHub - OpenFn/lightning: OpenFn/Lightning ⚡️ is the newest version of the OpenFn DPG and provides a web UI to visually manage complex workflow automation projects.

1 Like

@midigofrank i am using a prebuilt image which doesn’t have mix if i remember correctly

Ooh sorry, my bad.
Since you’re using the prebuilt images, you can migrate your db by running this command

docker compose build && docker compose run --rm web /app/bin/lightning eval "Lightning.Release.migrate()"

Thank you @midigofrank.
But, however, this seams to just be hanging without returning or displaying anything!

Kindly ignore this @midigofrank
I think its my docker that hangs sometimes. I have force restarted docker and seams i am now getting some results.

I am getting an ArgumentError below with docker compose build && docker compose run --rm web /app/bin/lightning eval "Lightning.Release.migrate()"

** (ArgumentError) could not call Module.put_attribute/3 because the module Lightning.Repo.Migrations.CreateJobs is already compiled
    (elixir 1.16.2) lib/module.ex:2360: Module.assert_not_readonly!/2
    (elixir 1.16.2) lib/module.ex:2041: Module.__put_attribute__/5
    lib/lightning-2.7.7/priv/repo/migrations/20220203064043_create_dataclips.exs:2: (module)

Eeeh! Now I think that’s being caused by the image.

@jnsereko can you try building the image locally and see if you run into the same problem

running docker compose build && docker compose run --rm web mix ecto.migrate at the home of the lightning repo returns

 => ERROR [web dev 20/20] RUN npm install --prefix assets                                                                                                                            0.1s
------
 > [web dev 20/20] RUN npm install --prefix assets:
0.072 rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
0.072  Trace/breakpoint trap
------
failed to solve: process "/bin/sh -c npm install --prefix assets" did not complete successfully: exit code: 133

@taylordowns2000 also doing something like below using docker compose returns the same ArgumentError. Could you know how to fix it?

web:
  platform: linux/x86_64/v8
  image: 'openfn/lightning:latest'
  command: ["/app/bin/lightning", "eval", "Lightning.Release.migrate"]
06:54:38.054 [error] Postgrex.Protocol (#PID<0.168.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (postgres:5432): connection refused - :econnrefused
06:54:38.054 [error] Postgrex.Protocol (#PID<0.167.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (postgres:5432): connection refused - :econnrefused
** (ArgumentError) could not call Module.put_attribute/3 because the module Lightning.Repo.Migrations.CreateJobs is already compiled
    (elixir 1.16.2) lib/module.ex:2360: Module.assert_not_readonly!/2
    (elixir 1.16.2) lib/module.ex:2041: Module.__put_attribute__/5
    lib/lightning-2.7.7/priv/repo/migrations/20220203064043_create_dataclips.exs:2: (module)

Hey @jnsereko, great news! I’ve managed to replicate the issue on my M1. I’ll experiment with different solutions and keep you posted once I find a working one.

1 Like

@midigofrank, could you be knowing how to fix this?

@jnsereko this is all related to the image. I found a similar thread here: Unable to compile default Elixir project from the getting started guide - #12 by Raf - Questions / Help - Elixir Programming Language Forum

1 Like

Oof!!, that actually worked.
I set the default platform to to amd64:

export DOCKER_DEFAULT_PLATFORM=linux/amd64

and modified Dockerfile-dev to include the flag ENV ERL_FLAGS="+JPperf true"

...
ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

FROM ${BUILDER_IMAGE} as dev
ARG NODE_VERSION
ENV ERL_FLAGS="+JPperf true"

...

Let me know if this doesn’t work on your end.

1 Like

thank you @midigofrank. This worked for me.

Below is my workflow

  • From my docker compose, web service, i added command: /bin/sh -c "/app/bin/lightning eval Lightning.Release.migrate && /app/bin/server" from the web service
  • I added ENV ERL_FLAGS="+JPperf true" to my .env that i supply to docker-compose

However i am now facing Could not check origin for Phoenix.Socket transport.
and a request to localhost:4000 hangs.

Origin of the request: http://127.0.0.1:4000

This happens when you are attempting a socket connection to
a different host than the one configured in your config/
files. For example, in development the host is configured
to "localhost" but you may be trying to access it from
"127.0.0.1". To fix this issue, you may either:

update [url: [host: ...]] to your actual host in theconfig file for your current environment (recommended)

pass the :check_origin option when configuring yourendpoint or when configuring the transport in yourUserSocket module, explicitly outlining which originsare allowed:

   check_origin: [" ",                  "//another.com:888", "//other.com"]

is this something worth worrying about?

Origin of the request: http://127.0.0.1:4000
This happens when you are attempting a socket connection to
a different host than the one configured in your config/
files. For example, in development the host is configured
to “localhost” but you may be trying to access it from
“127.0.0.1”. To fix this issue, you may either:

@jnsereko are you running the app in prod mode? (FYI, the prebuilt images are built in prod mode using the Dockerfile in the project root.)
If you’re running in prod, then you’ll need to configure some environment variables for the app to run smoothly. These environment variables are documented here: lightning/DEPLOYMENT.md at main · OpenFn/lightning · GitHub

For this particular error, the env var you’re looking for is ORIGINS, you’ll want to configure to ORIGINS=http://localhost:4000.

I think i am running it in development mode

Below is my env file. Note that i have ORIGINS=//localhost:*

EMAIL_ADMIN='admin@openfn.org'

SECRET_KEY_BASE=please_generate_a_more_secure_unique_secret_value_for_your_project

MIX_ENV=dev
NODE_ENV=development

URL_SCHEME=http
URL_HOST=localhost
URL_PORT=4000

LISTEN_ADDRESS=0.0.0.0
PORT=4000

ORIGINS=//localhost:*

WORKER_MAX_RUN_DURATION_SECONDS=60
WORKER_MAX_RUN_MEMORY_MB=500
WORKER_CAPACITY=4

MAX_DATACLIP_SIZE_MB=10

DISABLE_DB_SSL=true

POSTGRES_USER=hello
POSTGRES_PASSWORD=password
POSTGRES_HOST=postgres
POSTGRES_DB="lightning_${MIX_ENV}"
DATABASE_URL="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT:-5432}/${POSTGRES_DB}"

PRIMARY_ENCRYPTION_KEY=...
WORKER_RUNS_PRIVATE_KEY="..."
WORKER_SECRET="..."

DOCKER_RESTART_POLICY=no

DOCKER_WEB_HEALTHCHECK_TEST=/bin/true

LIGHTNING_EXTERNAL_PORT=4000

INIT_PROJECT_FOR_NEW_USER=false

PURGE_DELETED_AFTER_DAYS=7

ERL_FLAGS="+JPperf true"

also @midigofrank could you be knowing how to mount a project to OpenFn lightning running via docker at start up by supplying a config.json file and a project.yaml as docker compose volumes?