Advanced Namespace Tools blog

26 February 2017

Implementing /srv Namespaces Part Three

Even though the patch for private /srv namespaces was working right, there were a couple details that I felt weren’t quite right. One was the calls to srvclose that happened during process exit after the sgrp data structure was nilled out. The other was the fact that the base sgrp for process 1 was created on its first access to srv. I also felt when working in a private /srv namespace that I sometimes wanted to be able to share something back to a process in a different srvgroup, but without a shared /srv, the only way to do that was via /net listeners.

Setting Up a srv group for Process 1

This requires making a change to /sys/src/pc/main.c, and similarly for other arches. It’s just a matter of adding the expected code to userinit() in the same place that pgrp, egrp, fgrp, and rgrp are set.

   p->sgrp = smalloc(sizeof(Sgrp));
   p->sgrp->ref = 1;

Setting this up here means that it will no longer wait until process 1 accesses the srv device to assign this information, which means that branches could be trimmed from rfork handling in sysproc.c. The downside to doing this here is that it also needs to be added to pc64/main.c and similarly for main.c in every arch. At the moment, I only provide the modified configuration files for pc and pc64, so I followed that example and only provide the modified main.c in those subdirs of the frontmods dir. Anyone who decides to use ANTS on a different arch will need to add those lines to their arch’s main.c.

Rearranging sgrp = nil during Process Exit

The problematic sequence of events happened during pexit() in proc.c. I had added cleanup of the srv group pointer in similar fashion to how other similar resources were cleared:

   /* nil out all the resources under lock (free later) */
   qlock(&up->debug);
   fgrp = up->fgrp;
   up->fgrp = nil;
   egrp = up->egrp;
   up->egrp = nil;
   rgrp = up->rgrp;
   up->rgrp = nil;
   sgrp = up->sgrp;
   up->sgrp = nil;
   pgrp = up->pgrp;
   up->pgrp = nil;
   dot = up->dot;
   up->dot = nil;
   qunlock(&up->debug);

   if(fgrp != nil)
          closefgrp(fgrp);
   if(egrp != nil)
          closeegrp(egrp);
   if(rgrp != nil)
          closergrp(rgrp);
   if(sgrp != nil)
          closesgrp(sgrp);
   if(dot != nil)
          cclose(dot);
   if(pgrp != nil)
          closepgrp(pgrp);

The issue is that when closefgrp(fgrp) is called, it may be closing file descriptors from /srv, but the process sgrp has already been nilled out. This is what caused calls to srvclose within devsrv with up->sgrp as nil. The solution was to change the sequencing and wait to nil sgrp until after closefgrp() completes:

   /* nil out all the resources under lock (free later) */
   qlock(&up->debug);
   fgrp = up->fgrp;
   up->fgrp = nil;
   egrp = up->egrp;
   up->egrp = nil;
   rgrp = up->rgrp;
   up->rgrp = nil;
   pgrp = up->pgrp;
   up->pgrp = nil;
   dot = up->dot;
   up->dot = nil;
   qunlock(&up->debug);

   if(fgrp != nil)
          closefgrp(fgrp);
   if(egrp != nil)
          closeegrp(egrp);
   if(rgrp != nil)
          closergrp(rgrp);
   /* sgrp is nilled out here because closefgrp may need srvclose */
   qlock(&up->debug);
   sgrp = up->sgrp;
   up->sgrp = nil;
   qunlock(&up->debug);
   if(sgrp != nil)
          closesgrp(sgrp);
   if(dot != nil)
          cclose(dot);
   if(pgrp != nil)
          closepgrp(pgrp);

With this change made, the srvprocset() function I had added to devsrv to guard against nil pointers in sgrp became unnecessary. It was no longer being hit from srvinit() because process 1 was created with an sgrp, and it was no longer being hit from srvclose() during process exit because the sgrp pointer was maintained until after closefgrp().

Bringing Back a Global srv with devzrv

Using rfork V to enter a clean /srv namespace had been working as designed, but the absence of a global srv felt restricting in some cases. A sub-environment may wish to share pieces of its namespace via srvfs or provide execution resources via hubfs, and with no global srv available, this would require using aux/listen1 to share resources exclusively via /net.

The solution for maximum flexibility is to provide a duplicate #s device under the name of #z which maintains the previous /srv behavior and doesn’t split after rfork V. Amusingly enough, I had already prepared this as a patch months ago when first investigating independent srvs. Making a copy of the srv device under a different name was trivial: two functions needed a different public name (zrvname and zrvrenameuser) and they needed to be invoked as appropriate. For instance, srvname is called by devproc.c when printing information from the namespace file:

   if(strcmp(cm->to->path->s, "#M") == 0){
          srv = srvname(cm->to->mchan);
          if(srv == nil)
                 srv = zrvname(cm->to->mchan);
          i = snprint(buf, nbuf, "mount %s %s %s %s\n", flag,
                 srv==nil? cm->to->mchan->path->s : srv,
                 mh->from->path->s, cm->spec? cm->spec : "");
          free(srv);
   }

Note the use of the kernel-internal-label #M for the mount driver. What this code is doing is checking if a channel comes from the mount device, and if so, it asks devsrv to tell it the name of the service which is providing that chan. All I added was the additional check of zrvname if the first check of srvname returns nil. I can only admire the fearless use of the ternary within the snprint by the previous coder.