julia - Matplotlib not found when trying to run code in Linux cluster

Question

I am running a Julia script while ssh'ed into a Linux cluster. The code loads matplotlib as the following:

import Distributed   
using Distributed  

@everywhere  pushfirst!(Base.DEPOT_PATH,"/tmp/test.cache")   
   
@everywhere import PyCall   
@everywhere import PyPlot   
@everywhere pyimport matplotlib.pyplot as mpl 

My code works fine when I run it from the terminal. However, when I submit the job via Torque, the out file can be found here, and appears to suggest that the code cannot find matplotlib.

I'm a little confused by the above. Why does the code run normally when I run it in the terminal, but won't work if I submit a PBS file? Why can't it find matplotlib? It's using the python found in usr/bin/, which I believe is my default python. I've tried running pip, but it says matplotlib is updated. I've also imported other packages (like numpy and datetime) before I tried importing matplotlib, and there was no error. When I run Julia in terminal on the cluster and call PyCall, and then run

julia> @pyimport matplotlib

It loads completely fine. What am I doing wrong?

A related question (with my .pbs file) can be found here: Module FixedPointNumbers missing from the cache when running Julia on cluster .

EDIT: The one thing is that when I run Julia normally while ssh'ed into the cluster (i.e., just in the terminal, not by submitting the .pbs file), I get the error

Warning: No working GUI backend found for matplotlib

Could this result in the issue I'm having when submitting the job?

question from:https://stackoverflow.com/questions/65557577/matplotlib-not-found-when-trying-to-run-code-in-linux-cluster

Answer 1

The issue concerned the Python package I had linked to Julia. This version of Python didn't have matplotlib installed, and the only way for matplotlib to be properly installed on this version of python is to have admin privileges. The way I got around this was setting the Python environment in Julia to Julia's own Conda distribution:

julia> ENV["PYTHON"] = ""

I then installed matplotlib with Conda. Finally, I made sure I'm always using the same version of Julia in the .pbs file; i.e.,

module load julia/my_version

With these changes in place, everything appears to work fine.