Comment 1 Karl Mikaelsson 2017-09-18 16:05:31 CEST

One part of this is documentation.

Programs should accept "--help" arguments which would produce a summary of what the tool does and the arguments that influence its behavior. (Bug 2072)

In-depth documentation about each command could be provided by manual pages. This information is delivered through the Administrators Guide today, but the TAG is delivered in formats that aren't very suitable to shell users.

Comment 2 Karl Mikaelsson 2017-09-18 16:06:51 CEST

Additional ease of use could come from having the shell automatically complete the options and arguments for our tools. tl-config could for example complete the names of the configuration parameters to make it easier for the administrator to find the proper parameter to act on.

Comment 4 Karl Mikaelsson 2018-06-28 11:12:22 CEST

To minimize information duplication, I've settled for using DocBook to write the manual pages. The reference/refentry tags are very well suited for this purpose.

A problem is that our current cenbuild lacks a good way to convert DocBook XML to man pages. docbook2man from cendio-build-docbook-utils works just fine for SGML but barfs on XML files. It also requires modifications to cendio-build-opensp (missing nsgmls->osgmls link) and the addition of a cendio-build-perl-SGMLsp module to work properly.

As an alternative, docbook2X also provides a docbook2man that works much better with DocBook XML. I have a git branch of cenbuild that includes docbook2X and a few required Perl modules:

 http://git.cendio.se/cgit/~derfian/cenbuild.git/commit/?h=man-pages&id=4b7cf3ff4812c3384484c7f495e8da7f4b1de6ec

Comment 5 Karl Mikaelsson 2018-06-28 14:14:27 CEST

I'm not going to have time to finish this before heading off to
vacation, so I'll report on what remains to be done for the manual page side of things:

- DocBook conversion of existing manual pages:

  The decision to switch to DocBooc was made late in the
  process. There were 23 manual pages already written, and I've
  finished fully converting 4 of them to DocBook refentrys. This
  is mostly straight-forward but time-consuming.

  A very Work-In-Progress git branch can be found here:

    http://git.cendio.se/cgit/~derfian/thinlinc.git/log/?h=man-pages


- Integrate manual pages into ThinLinc Administrator's Guide

  My idea is to include these manual pages into the TAG,
  effectively replacing the command reference with a separate
  reference part of the documentation.

  Some work on this has been done (see the Make rules for
  pdf/reference.pdf). I'm not convinced this is the way forward,
  and I might have to revert back to the custom
  include/preprocessor way of merging DocBook documents used by
  the TAG to make it work.


- Packaging and integrating manual pages with man on the server

  The manual pages needs to be installed on the system to be
  useful. I haven't spent much time figuring out which package
  these new files should be a part of.

  Depending on where we decide to install our manual pages, we
  might have to edit MANPATH to have the manual pages available
  to the end user. How we should do this on different platforms
  require more investigation.

Comment 6 Pierre Ossman 2018-07-03 10:41:38 CEST

There is no clear small step here. We will have to continue this work during a later batch.

We should also split out the man-pages and bash completion to separate bugs so we can track each feature separately.

Comment 7 Karl Mikaelsson 2019-01-15 17:13:06 CET

(In reply to comment #4)
> http://git.cendio.se/cgit/~derfian/cenbuild.git/commit/?h=man-pages&id=4b7cf3ff4812c3384484c7f495e8da7f4b1de6ec

This branch has been rebased onto the latest master and the commit hash changed. You'll find the commit at this URL:

http://git.cendio.se/cgit/~derfian/cenbuild.git/commit/?h=man-pages

Comment 9 Pierre Ossman 2022-02-03 10:13:51 CET

Modern systems don't use $MANPATH, instead having the single configuration file /etc/man_db.conf (or /etc/manpath.config). This makes things tricky for us as we don't want to magically modify a system configuration file.

Fortunately $MANPATH is clever enough that you can use it to just add things, rather than completely overriding the default configuration:

> If $MANPATH is set, manpath displays its value rather  than  de‐
> termining  it  on  the fly.  If $MANPATH is prefixed by a colon,
> then the value of the variable is appended to  the  list  deter‐
> mined from the content of the configuration files.  If the colon
> comes at the end of the value in the variable, then  the  deter‐
> mined  list  is appended to the content of the variable.  If the
> value of the variable contains a double colon (::), then the de‐
> termined  list  is  inserted in the middle of the value, between
> the two colons.

So if we set MANPATH=:/opt/thinlinc/man then we should just add our man pages to the system ones.

I suspect that searching our man pages is still broken though as that is done via a special job and put in to an index database.

Comment 10 Pierre Ossman 2022-02-10 13:58:23 CET

We've now switched from DocBook to reStructuredText and Sphinx (bug 4951). However Sphinx does have a man page builder that seems to produce sufficient results.

Comment 11 Pierre Ossman 2022-02-10 13:59:18 CET

A problem is how these things should be installed as we currently don't install any documentation (bug 1161), and doing so might require some build system changes (bug 4885).

Comment 12 Pierre Ossman 2022-03-11 17:11:29 CET

I found a formatting bug in Sphinx' man page builder unfortunately:

https://github.com/sphinx-doc/sphinx/issues/10259

If you use the "option" directive then the formatting is just all bold and doesn't follow the style man pages are supposed to have.

Probably not a major issue, but annoying.

We could avoid using the "option" directive, but the end goal was to have it help out in cross referencing things (bug 7783).

Comment 13 Pierre Ossman 2022-03-14 10:33:24 CET

(In reply to Pierre Ossman from comment #9)
> 
> So if we set MANPATH=:/opt/thinlinc/man then we should just add our man
> pages to the system ones.
> 

We already set MANPATH under some conditions, and have been seen 2004 (so basically every version of ThinLinc). We did tweak it a bit in bug 2056 though.

Oddly enough, everything work fine if I remove MANPATH completely. It seems there is some magic in man that guesses at paths based on PATH. If I remove all /opt/thinlinc stuff from the PATH then man no longer finds our pages.

This is normally controlled by the MANPATH_MAP setting. But we don't add any such settings, so there seems to be some built in magic as well.

> I suspect that searching our man pages is still broken though as that is
> done via a special job and put in to an index database.

It would seem so. Both "whatis" and "apropos" fail to find our pages. Not sure exactly how this caching is done, except that it is mandb that does it.

Comment 14 Pierre Ossman 2022-03-14 10:50:45 CET

(In reply to Pierre Ossman from comment #13)
> 
> This is normally controlled by the MANPATH_MAP setting. But we don't add any
> such settings, so there seems to be some built in magic as well.
> 

Indeed. I can find this comment in the source code:

> /*
>  * For each directory in the user's path, see if it is one of the
>  * directories listed in the man_db.config file.  If so, and it is
>  * not already in the manpath, add it.  If the directory is not listed
>  * in the man_db.config file, see if there is a subdirectory `../man' or
>  * `man', or, for FHS-compliance, `../share/man' or `share/man'.  If so,
>  * and it is not already in the manpath, add it.
>  * Example:  user has $HOME/bin in his path and the directory
>  * $HOME/man exists -- the directory $HOME/man will be added
>  * to the manpath.
>  */

So it is translating /opt/thinlinc/bin => /opt/thinlinc/bin/../share/man, which is indeed the right place for our man pages.

This was added in 2009, which explains why we were previously fiddling around with MANPATH. It seems that is entirely redundant in modern systems.

Comment 15 Pierre Ossman 2022-03-15 08:26:56 CET

References also seem to have some problems. It just renders the internal name when creating the man page. For other writers the title of the reference is used, which is much more sensible.

Note that we need to rethink references anyway since they make much less sense in the man pages where the rest of the TAG isn't readily available.

Comment 16 Pierre Ossman 2022-03-15 08:28:24 CET

Will attempt to do this now in order to provide a sensible place for the tlctl documentation (bug 3707).

Only the man pages will be done now. Everything else will be split of to separate bugs.

Comment 17 Pierre Ossman 2022-03-16 16:09:40 CET

(In reply to Pierre Ossman from comment #15)
> References also seem to have some problems. It just renders the internal
> name when creating the man page. For other writers the title of the
> reference is used, which is much more sensible.
> 
> Note that we need to rethink references anyway since they make much less
> sense in the man pages where the rest of the TAG isn't readily available.

Reported this upstream and made an extension that has some sensible default behaviour:

https://github.com/sphinx-doc/sphinx/issues/10269

Comment 18 Pierre Ossman 2022-03-16 16:34:36 CET

More bugs:

https://github.com/sphinx-doc/sphinx/issues/4371

Sharing the doctree cache with the man page builder is breaking the html and pdf one. :/

Comment 22 Pierre Ossman 2022-03-17 15:34:44 CET

An issue in this is how things should be presented in TAG. Previously we had everything on a single page as most commands just had a brief description. With this change each command is described in much more detail and is better suited for one page each (which is what the HTML output does, and straight forward to do with the PDF output).

The question is what the main "commands" page should look like.

Either it is just a brief introduction, followed by individual pages for each command. You would then need to go through them or search to find something specific.

Or we construct some overview of the different commands, e.g. as a bullet list or definition list. The main downside would be that we would need to manually duplicate things. It might also be odd that this chapter has such a "mini-table of contents", but no other.

(There is also a problem with linking to the commands. We haven't fixed cross references (bug 7783), and :doc: references are broken in rst2pdf¹.)

¹ https://github.com/rst2pdf/rst2pdf/issues/1065

Comment 23 Pierre Ossman 2022-03-18 14:17:45 CET

I've chosen to omit documentation for our services. They still live in /opt/thinlinc/sbin, as that's where daemon binaries customarily live, but they aren't supposed to be directly executed under any circumstances. They also have no options.

Comment 35 Pierre Ossman 2022-03-21 08:08:37 CET

Everything should now be in place, so this should finally be complete.

> - All commands that users or administrators are supposed to directly execute should have man pages

Yup. And correctly categorized to section 1 or 8 depending on if it's a user command or not. (after bug 7868 at least)

> - Our man pages should be available to users without specifying the full path and without manual configuration

Works fine. Needs a logout and login after installing though to get our things in $PATH.

> - Developers should not be forced to update two places to have the man pages and TAG in sync

The main information now comes from a single source. There is some annoying duplication when adding a new page though...

Comment 43 William Sjöblom 2022-03-23 10:28:20 CET

I have now tested the acceptance criteria:

• ☑ All commands that users or administrators are supposed to directly
  execute should have man pages

  Yes!

• ☑ Developers should not be forced to update two places to have the man
  pages and TAG in sync

  Yes!

• ☐ Our man pages should be available to users without specifying the
  full path and without manual configuration

  On both Ubuntu 18.04 and Fedora 35, this is the case. Sadly, this
  does not seem to be working on SLES12.

I have compared the TAG (HTML, PDF) and man-pages and everything looks
good and consistent between the three.

Additionally, I have also done some proofreading of the man-pages. As a
result of this, I have done some minor tweaks fixing the errors and
inconsistencies I found. These are very minor and should not need
additional testing.

Marking as REOPENED due to the man-page issue on SLES12.

Comment 45 Pierre Ossman 2022-03-24 08:31:36 CET

Damn SUSE. They are setting $MANPATH to the output of `manpath`, which generally is a no-op. But it has the effect of locking the man path to whatever was found just after the shell was started. So any subsequent changes to $PATH will be ignored.

Not sure why they do this. They've done this since day one and the code has just been updated to continue working with modern versions of man-db.

Need to check how SLE 15 behaves.

Comment 46 Pierre Ossman 2022-03-24 09:19:59 CET

Unfortunately the same thing is still true on SLE 15.

I've contacted SUSE to see why they have this and if it could be removed in the future. But for now it looks like we'll have to concoct a workaround.

Comment 48 Pierre Ossman 2022-03-24 10:56:34 CET

$MANPATH is now updated properly on SUSE. It also remains unset on other distributions.

Tested on SLES 12, SLES 15 and Ubuntu 20.04.

Comment 49 Linn 2022-03-24 14:40:01 CET

Checked the code for the SUSE workaround, and can confirm that we still use the same logic for the MANPATH as before we started working on this bug.

Also checked the code for the minor tweaks done to some manpages (see comment 43), and they all look appropriate.

Lastly, I tested the relevant acceptance criteria on RHEL 8 and SLES 15:

> - All commands that users or administrators are supposed to directly execute should have man pages
Yes, tlctl, tlctl session and tlctl load all have manpages.


> - Our man pages should be available to users without specifying the full path and without manual configuration
They are, but as mentioned in comment 35 you need to log in and log out for them to work properly.