Right. Thanks. Can you hear me?
Yeah, unified storage. When we talk about unified storage, we're going to be, I'll be talking about
IBM's, elastic storage system, which internally we refer to as ESS. And when we talk about unified
storage or moving towards unified storage, it hasn't always actually been diversified. I think a lot
of
projects in TSD, they have only always known unified storage because presently this applies to
non-hpc see projects and non-hpc projects, only interact with our current storage system which is
called HNAS, that may not mean a lot to to users but basically it's the system that holds the
durable partitions and the home partitions, and it's a system that all our servers talk to. Some of the
service you heard about today. Like
publication, data import etc., they all talked to HNAS us at the moment. And if you log into
your project VM's, either Linux or Windows, then that is the system you're talking to when you're
storing your data on the durable partition.
It has been a hybrid old and new setup for the HPC projects and it has always been that way,
because in the past we used to have GPFS system, but about two years ago we moved to this new
ESS system and is a system is what is connected to Colossus, and for that it hosts the cluster
partition but the project VM´s didn't have access to that system, and don't have yet, and for
those projects we setup dedicated submit hosts, and submit hosts are the only host that talked to
both the new ESS and talk to the durable partitions. And it has created a bit of a problem because
the people that want to run analysis on Colossus currently had to sort of import the data into to
TSD, store the data on HNAS, and then I have to copy it over before I submit host to ESS, and that
created a lot of overhead for users and also created a lot of overhead for us in order
to like maintain two systems.
So what we're talking about when we're moving to a unified storage system, is that we're
talking about the migrating to the unified ESS, and in the future that's going to apply to all
projects who started migrating the first move project and what it means when we talk about unified
storage is that all our services or our VM´s, Colossus etc., will all be talking to that same file system.
And at the same time, the thing that we're doing is that we're sort of merging the
durable home and close to partitions into a single single file system. So basically what that
means for you as a user is that you will see certain improvements. One of them is, like I said, is
unified durable home and cluster. There will be no more need for copying back and forth data
between durable and cluster, which
mostly applies to the projects are currently at HBC, and it also in means for us that we no
longer have to be patient because people will be storing data on durable copy it to Cluster,
store it there, modify it and then copy it back. So the overall need for storage will go down slightly.
But we also did, since we're merging home, the durable and cluster into sort of a same
file system. We could also move it into the same directory tree. So all the new directories will be
actually stored under a common rule, which is /tsd/pXX/, which is probably known to be
pretty much all projects at the moment.
What we did when we moved from the old system, HNAS to ESS, is that we improve the security of the
protocol we´re using, particularly the NFS protocol, and what we added was
Kerberos, which is an improved security feature using authentication and, it basically means
that when you connect to the submit host or to the new systems, you'll be
authenticated yourself with a password and you will be granted a ticket with a certain lifeline,
and and during that lifetime, you'll be allowed to talk to the system, and when the ticket expires
you will have to re-authenticate. So the security above and beyond what we already offered in TSD.
And finally, it's a feature for us, but also for you, is that it creates a simplified
infrastructure. We now have a system that talks to all the machines though, it is a storage system
that sort of bundles all the protocols that we use. So GPFS for the computers clusters, NFS for the
Linux hosts and SMB for the windows machines. So that
means less overhead for us. And at the same time, it's massively scalable
And we've seen massive data needs increases over the last two years, and we expect that to continue
and this new system should be able to handle that much better.
Then what does it mean? The migration procedure for you as a user?
What we'll do, we'll send out emails to the PI with a migration date. And the idea is that we will
have a downtime between 3:00 in the afternoon until 10:00 in the morning. So we can do a lot of
immigration work outside of office hours, to minimize the impact.
The PI will probably have to forward the email to you because we don't host all the emails for the
users. And then, you'll be asked to save any work prior to the downtown and then log off.
You will not be able to use or access TSD during migration. So you cannot log into your
Linux and windows machines, you will not be able to process data on HBC but what is
not affected is our the Nettskjema deliveries, so data collection will continue as is typical
still use publication and some portals.
Then what you do is that you look back in the next morning and the idea is that
everything should just work. It was put in a lot of efforts to make all the services talk to both
systems and during the migration will have phases where some project will only be talking to HNAS,
like the non-HBC projects. Some of them will still be in the state where they talk to
HNAS and ESS, like the HPC projects,
and then there will be a phase, at the same time, there will be project that only talk to users
and we have modified our services accordingly to be able to handle bar.
It's probably not answering all your questions, if you have questions ask them now or address them to
our dedicated queue to not overwhelm the helpdesk and that's tads∂-ess-migration@usit.uio.no
I guess I could take a few questions now.
Thank you very much. There are probably some questions yet about this. We will try to take more
questions and answer them at the end where we assert the Q&A. So many questions will be over there
because I'm now just going to move on to do the launch of Ecucloud.