Topic: NEMO Parallel Error When Running in EnvironmentSubscribe | Previous | Next
I'm running NEMO on my machine (Mac OS X 10.15.7, Julia 1.5.3). I'm using a Julia environment since there are some backwards compatibility issues with some requirements. The quick start example works fine if I specify nproc = 1, but when I run it with the default (nproc = 0), I get the following error:
Using 6 processes for parallelized operations.
NEMO encountered an error with the following message: On worker 2:
ArgumentError: Package NemoMod not found in current path:
- Run `import Pkg; Pkg.add("NemoMod")` to install the NemoMod package.
require at ./loading.jl:893
eval at ./boot.jl:331
#106 at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Distributed/src/process_messages.jl:294
run_work_thunk at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Distributed/src/process_messages.jl:79
macro expansion at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Distributed/src/process_messages.jl:294 [inlined]
#105 at ./task.jl:356.
To report this issue to the NEMO team, please submit an error report at https://leap.sei.org/support/. Please include in the report a list of steps to reproduce the error and the error message.
It seems to me that there might be an issue with the workers recognizing the environment or finding NemoMod when more than 1 processor is specified, but I'm not sure. Is there an obvious fix to this anywhere?
Thanks!
James
This is indeed a case that NEMO isn’t set up to handle. We could consider adding an environment argument to the calculatescenario function, but my sense is it would be rarely used (I’d appreciate hearing others’ views on this, though!).
Since you’re running NEMO from a Julia prompt anyway, you should be able to work around the issue by instantiating processes yourself and activating your target environment in them. For example:
julia> using Distributed
julia> addprocs(max(div(Sys.CPU_THREADS, 2), 1))
julia> @everywhere begin
import Pkg
Pkg.activate([path_to_your_environment])
end
This will set up the processes in advance, and NEMO should use them when you run calculatescenario with numprocs=0.
I hope this helps!