Should I just use that microphone?
I'm Maiken, I'll show you a small demonstration about getting access to the light weight HPC and the ML nodes
that Jon earlier talked about, and Anne also. I'll start sharing my screen
If I can get the right screen, this one.
I'll do this one and I'll move my things just a second... So can you see now
the terminal and a webpage? Yes, good. First of all, it was mentioned that Abel shut down
as some of, you know. And then there was like a little bit of a vacuum because
there were no easy access local resources at UiO, and that's why we quickly
initiated.. got in place the lightweight HPC and the ML nodes and there's still
in the state that they were then, eventually it will be incorporated into Fox and Educloud system
But right now they are stand alone resources. The whole point is that they are easy to get access to and there are
machines, single machines that you can test out data, test out your computation and so on. So, first of
all, how to get access
In the UiO pages, in the services and tools and IT services you get a menu with
the different services that are provided. You will go to the computation tab and here you see
the services that have been talked about in this conference, Sigma 2, TSD, Educloud
and so on. If you scroll a bit further on you see free for all computation and free for all machine learning. And those are the two I will show you now.
In both of these, there is the same Nettskjema that you use to get access. So I'll show you
for the lightweight HPC. You click in there, and then you see here, get accessed by filling
in Nettskjema. First of all I'll just stop a little bit on this page and show you a
little bit about the information here. So we have the different machines listed here so there's like
single machines as I said, Freebio, Freebio2.
Those are ARM machines, so the ARM architecture. And then we have bioint machines
from here, that are AMD EPYC machines and Intel machines. So you see there's different types
of machines and maybe you're interested in just checking your software on different architectures.
Or, you know, you just want to be able to run quickly on whatever machine. So you can choose one of these
and the same for the ML nodes, there is an overview. If you go to the ML nodes page of the different
types of machines, here are the specifications and here is also a link to Nettskjema. Let's click on Nettskjema
Okay, this is how it looks. Request access to lightweight HPC and ML nodes. You will have to select
which ones you want access to. You can fill it out twice and ask for HPC and then next time ML
nodes, that's completely fine. And that's basically it, there's a link to the documentation here.
We process this request very quickly. Typically you might have to wait about an hour for the access to be ready.
But that is the max time mostly to get everything propagated, and once you've got access, you will get
an email to tell you that that's ready and that you can log-in.
So how to login, you open a terminal and SSH into the machine as you've done as, I'm sure, many times on
other machines. So let's check, let's say, I want to go into view in 01.
Then I at the moment, just SSH directly into Bionic, bioint01.hpc.uio.no
There is information here about exactly what SSH command to do, if you click on
the link here, you see it here. and that's what I have to do, and since I am already maikenp
on my laptop here I don't have to say,
maikenp@ here, but with another name you would have to have to do that
I'll just do that for the demonstration.
You use your UiO login and password to go in. And now I am in. You will see your own home
area and that home area will be the same home area on all the light HPC machines, but it will
be different from your UiO home area.
The same goes for the ML nodes, you will have the same home area of all the ML nodes, machine
learning nodes, but it will be a different home area than the light HPC, and also than the UiO
your usual UiO area.
So, this is my messy home area.
On all these systems the lightweight HPC, the ML nodes and also on the
national resources and everything, there is this module system so it's the same here. If I do module
avail python for example. I will get a list of the different python modules, but just module...
module avail general
You'll just see a list of all the different modules. So that means all the different software that
is installed here. And as as was mentioned, many times before also, for the different other
services, if there are, if there's other software that you want you contact us
to get help to install those, and we use the same easy build and module system as in the national resources.
And on Fox.
I don't know if I should, I can just use.. I'll show you a module list. I have no modules
loaded. And module Avail python
If I say python now, you see, there's no command found, let's say, I want this python module
Module load python 395, if I now write python you see that that the python 395 is loaded.
Module list, you can see also with module list all the other packages that are included in
this one, the composite python GCC core package. And if I do module purge, they're gone again
if I do python, there's nothing and so on. I can load python 2.7 if I want and etc.
Was there anything more I wanted to say?
Can you say anything about what the limits of the resources are? Can you check how much resource data is
available at this time? Well, yes and no. So for the ML nodes it is very easy and will implement this
also in the lightweight nodes. Let me log into ML
Since these are single machines it is a bit more difficult to get to the global overview, in Fox you
can use squeue or, you know, in the national resources, you can use squeue command and see whatever is
running. So for the ML nodes, there has been implemented a quick check to see the status of the
different nodes. You can see which one is used more or less, and typically you would then maybe choose
the one that's less used, and this kind of easy snapshot overview will also be implemented in
the lightweight HPC so you don't have
log into each of the lightweight HPC is to see, you know, which one is the one least used
at the moment you can just use top and filter for normal users, which I didn't do right now. Then you can
see if it's really busy or not. Is this one now using 100% CPU on Matlab?
Yes, but only for that CPU and there's many
CPUs. So typically 128 or something like that, but that Matlab is using just 2
Anything more?
So this example you were running at this time, was running Matlab on two processors? Yes
Do they get any more resources from this? Why not programming in parallel..
Well actually you touched upon something very important here, which I forgot to mention and
it's very important when it comes to the light HPC and ML nodes, this is really, there's no
queuing system, so in principle, one user could just
use all the.. set up a process or run the program that uses all the processors
on the whole machine. It's quite important here to try to be.. so there's anarchy basically
it's quite important that the users themselves try to think "okay I'm using all the resources here
maybe I should share with the others", you know be a bit
considerate about how much you're running and if you find yourself needing to run a lot, you rather
asked to go to Fox or to the national resources. So this is like a testing, more of a
testing area or running a smaller jobs. so long and so on. And but yeah, you can
run many processes. If you set up an MPI job, you will run over the process.
And if you have a job that only is able to run on one processor, then you do you can work on your computer
Exactly, but if you want like 20, or 50, this is fine to use, or actually I don't
know what would be the limit for what is fine or not compared to how many is using it
But this example that someone is running on one processor, 100% CPU on Matlab, and it probably takes a couple
of hours, then it's probably better to use this. Yes and maybe you don't have
Matlab yourself, maybe you haven't managed to set it up so then you can use this, maybe you don't have battery
to run all these jobs locally. Yes, things like that. You have the software, lots of software
available. What are the most common jobs to run on these computers? We saw Matlab
and I assume R jobs as well. I assume, but I really don't know, I haven't got over, you typically the
scientific software, so R and Matlab and probably some
Bioinformatics things and but really I don't have the answer to that Can you run SPSS on this
What's that? SPSS, no that's Windows.. No Windows, Linux based only.
Thank you, Maiken