NethServer Documentation Release 0

Nethesis

January 19, 2017

Contents

1

2

3

4

5

Rules and conventions 1.1 Introduction . . . . 1.2 Development process 1.3 RPM package rules . 1.4 Internationalization .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

3 3 3 8 10

Architecture 2.1 Databases . . . . . 2.2 Templates . . . . . 2.3 Actions and events 2.4 Services . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

13 13 17 23 28

Implementation 3.1 Filesystems options . . . . . . . . . . . 3.2 DNS . . . . . . . . . . . . . . . . . . . 3.3 DHCP . . . . . . . . . . . . . . . . . . 3.4 Log retention and rotation . . . . . . . . 3.5 Network . . . . . . . . . . . . . . . . . 3.6 Auto-generated random URL . . . . . . 3.7 Migration from NethService/SME Server 3.8 Certificate Management . . . . . . . . . 3.9 Yum plugin . . . . . . . . . . . . . . . . 3.10 Backup . . . . . . . . . . . . . . . . . . 3.11 Firewall and Gateway . . . . . . . . . . 3.12 IPS (Intrusion Prevention) . . . . . . . . 3.13 Samba . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

31 31 31 33 34 34 38 39 41 42 43 47 57 59

Web interface 4.1 Web interface . . . . . . . 4.2 Creating web UI module . 4.3 Dashboard . . . . . . . . 4.4 Customization . . . . . . 4.5 Creating inline help pages 4.6 TODO API . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

65 65 66 68 72 72 74

Packages 5.1 Building RPMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Building ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

77 77 80

. . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

i

5.3 5.4 6

Repositories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Package groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

83 83 89 96 96 98 99 100 101 102 103 104 105 106 109 110 114 114 115 115 115 116 117 118 118 124 125 126 126 127 127 127 128 128 128 129 130 130 133 134

7

Appendix 7.1 License Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Rebranding Administrator Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

137 137 138 139

8

Indices and tables

141

ii

Modules 6.1 Directory (LDAP) . . . . . . . 6.2 Email . . . . . . . . . . . . . . 6.3 Chat . . . . . . . . . . . . . . . 6.4 FTP . . . . . . . . . . . . . . . 6.5 UPS . . . . . . . . . . . . . . . 6.6 TFTP . . . . . . . . . . . . . . 6.7 Proxy POP3 . . . . . . . . . . 6.8 POP3 connector . . . . . . . . 6.9 ownCloud . . . . . . . . . . . . 6.10 Webmail (Roundcube) . . . . . 6.11 Collectd . . . . . . . . . . . . . 6.12 Phone Home . . . . . . . . . . 6.13 Web proxy (Squid) . . . . . . . 6.14 Web antivirus . . . . . . . . . . 6.15 Web content filter . . . . . . . . 6.16 Web proxy report (LightSquid) 6.17 WebVirtMgr . . . . . . . . . . 6.18 Disk analyzer (Duc) . . . . . . 6.19 SNMP . . . . . . . . . . . . . 6.20 Tomcat 7 . . . . . . . . . . . . 6.21 PostgreSQL . . . . . . . . . . . 6.22 UnixODBC . . . . . . . . . . . 6.23 WebTop 4 . . . . . . . . . . . . 6.24 VPN . . . . . . . . . . . . . . 6.25 Unbound . . . . . . . . . . . . 6.26 Ntopng . . . . . . . . . . . . . 6.27 Samba-audit . . . . . . . . . . 6.28 Redis . . . . . . . . . . . . . . 6.29 Memcached . . . . . . . . . . . 6.30 SMARTD . . . . . . . . . . . . 6.31 CUPS . . . . . . . . . . . . . . 6.32 Avahi . . . . . . . . . . . . . . 6.33 Clamd . . . . . . . . . . . . . 6.34 c-icap . . . . . . . . . . . . . . 6.35 IAX modem . . . . . . . . . . 6.36 MySQL . . . . . . . . . . . . . 6.37 Hylafax . . . . . . . . . . . . . 6.38 NTPd . . . . . . . . . . . . . . 6.39 HA (High Availability) . . . . .

80 81

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

NethServer Documentation, Release 0

NethServer is an operating system designed for small offices and medium enterprises. It’s simple, secure and flexible.

About • • • • • • •

Official site: http://www.nethserver.org Bug tracker: http://dev.nethserver.org Twitter: @nethserver Source code: http://dev.nethserver.org Mailing list: [email protected] ML archive: Google Groups IRC: #nethserver on freenode.net

Contents

1

NethServer Documentation, Release 0

2

Contents

CHAPTER 1

Rules and conventions

1.1 Introduction NethServer is an Open Source operating system for the Linux enthusiast, designed for small offices and medium enterprises. It’s simple, secure and flexible. NethServer is ready to deliver your messages, to protect your network with the built-in firewall, share your files and much more, everything on the same system.

1.1.1 Target audience This manual is intended for developers who are interested about NethServer internals. Reading this manual you should be able to learn how to extend, enpower and debug the NethServer platform.

1.1.2 Get involved The NethServer project welcomes anyone who would like to become involved in the project. This is a brief list of things to do (in a sparse order): • Writing documentation • Bug reporting, bug triaging, QA testing • Translating the web interface • Developing / coding in Perl, PHP, Python, Bash Note: This manual is a work in progress. If you can’t find some information please also check the developer wiki: http://dev.nethserver.org/projects/nethserver/wiki/Packages Always feel free to ask in ML or IRC channel!

1.2 Development process 1.2.1 Issues An issue is a formal description of a known problem, or wished feature, inside a tracker. There are 4 types of issue:

3

NethServer Documentation, Release 0

• Bug: describe a defect on the software, it must lead to a resolution of the problem. For example, a process crashing under certain conditions • Feature: describe a new wished function inside the software. For example, a new GUI interface to configure user preferences • Enhancement: describe a small improvement of current code or features. For example, remove harmless warning of a running process • Task: describe a generic task not strictly related to source code. For example, a document about a new feature Bugs, features and enhancements will always produce a commit inside a SCM repository and/or a new software package (typically RPM) containing the new code. Developer must take care to link commits to code reporting the commit inside the issues, and the issue number inside the commit comment. All released packages must contain a list and a description of related closed issues. Do I need to open a new issue? Yes, if what you’re talking about will produce some code. By the way, it’s perfectly reasonable to not fill issues for occasional small fixes, like typos in translations. Issues are not TODO list. Issues track status changes of a job, to output of the job will be a new object implementing the issue itself. If you are exploring some esoteric paths for new feature or hunting something like an heisenbug , please write a draft wiki page with your thoughts, then create a new issue only when you’re ready to write a formal description and produce some output object. A process for a new feature, can be something like this: • Make informal discussion using small meetings or ML discussions • Create a wiki page with notes and thoughts (team contributions are welcome!) • When the wiki page is pretty “stable” and the whole thing is well outlined, a team member will create one or more new issues. • If the wiki page is a feature design document, the feature can simply point to the wiki page • The wiki page should become a technical documentation of the feature, or a changelog for a new release At any point in time, make sure the issue status reflects actual work. Writing issues How to write a bug report: • Point to the right component with the associated version • Describe the error and the expected behavior • If possible, suggest a fix or workaround • If possible, add a piece of system output (log, command, etc) • Text presentation matters: it makes the whole report more readable and understandable How to write a feature or enhancement: • Everybody should understand what you’re talking about: describe the feature with simple words using examples • If possible, add links to external documentation • Text presentation matters: it makes the whole report more readable and understandable More information: 4

Chapter 1. Rules and conventions

NethServer Documentation, Release 0

• https://developer.mozilla.org/en-US/docs/Mozilla/QA/Bug_writing_guidelines • http://fedoraproject.org/wiki/Bugs_and_feature_requests • http://fedoraproject.org/wiki/How_to_file_a_bug_report Issue status Issues can have multiple states. Each state should be handled by a person who fit in the below roles. Of course, one person can wear one or more roles. See also the workflow figure on FedoraProject.org. Triager

The Triager handles all issues in NEW state. She • collects missing info, setting the NEEDINFO flag • sets state to TRIAGED when the issue is clear enough and all requirements are discovered. If the issue is a Bug, she can also change the status to CLOSE and set resolution: DUPLICATE, INSUFFICIENT_DATA, NOTABUG. Additional infos are appreciated. If the issue is NOT a Bug, sets the generic REJECTED resolution, specifying the reason in a comment. Developer

The Developer • Takes a TRIAGED issue and put it ON_DEV setting the Assignee to himself, • Writes test cases, optionally annotating RPM changelog message for Packager, or • Writes and updates the documentation associated with the code. • Finally pushes the code/docs changes to SCM • Changes the issue state to MODIFIED, resetting the Assignee. If the issue is a Bug, he can change the status to CLOSE and specify a Resolution: CANTFIX, WONTFIX, WORKSFORME, CURRENTRELEASE, NEXTRELEASE, UPSTREAM. Additional infos are appreciated. If the issue is NOT a Bug, specify the generic REJECTED resolution, specifying the reason in a comment. Packager

The Packager: • Pulls changes from SCM and builds the RPM. • Uploads the RPM to the testing repository. • Changes state from ON_DEV to ON_QA, specifing the RPM file name and yum repository name in the issue comment. • Takes care to write a test case (or ask to a developer), if the test case is missing. When the package is VERIFIED from the QA team, the Packager • Commits a release tag (using release-rpm command from nethserver-mock).

1.2. Development process

5

NethServer Documentation, Release 0

• Re-builds the tagged RPM. • Uploads the RPM to updates (or base) repository. • Checks yum update works fine then pushes the tagged commit to SCM. • Finally, sets state to CLOSED (with blank Resolution field), adding a comment to the issue, containing the RPM file name and the yum repository name where to find the package. When the package is CLOSED, all related documentation must be in place. QA team member

The QA team member • Takes an unassigned issue ON_QA state and sets the Assignee field to herself. • Tests the package, following the test case documentation written by the Developer • She can set NEEDINFO flag if informations about how to test the code are missing. • When test is passed she sets the issue state to VERIFIED, otherwise she puts it back in TRIAGED state cleaning the Assignee field.

1.2.2 Version numbering rules NethServer releases bring the version number of the underlying CentOS. For example NethServer 6.4 beta1 is based on CentOS 6.4. Packages have a version number in the form X.Y.Z-N (Eg. nethserver-myservice-1.0.3-1.ns6.rpm): • X: major release, breaks retro-compatibility • Y: minor release, new features • Z: bug fixes/enhancements • N: spec modifications inside the current release

1.2.3 Commit message style guide Commit messages must include four components • WHERE • WHAT • WHY #Num (see http://www.redmine.org/projects/redmine/wiki/RedmineSettings#Referencing-issues-incommit-messages) • WHY Name See also jQueryUI Commit message style guide: http://contribute.jquery.org/commits-and-pull-requests/#commitguidelines. Example: git commit createlinks -m “createlinks: add nethserver-myserver event. Refs #1234” Refs links the commit to a Redmine issue.

6

Chapter 1. Rules and conventions

NethServer Documentation, Release 0

1.2.4 Documentation The developer must take care to write all documentation on: • wiki page during development • Developer Manual before release • Administrator Manual before release • Inline help before release Packages should be inside testing repositories untile all documentation is completed.

1.2.5 ISO releases 1. An ISO release starts whenever a target version is reached 2. Search for all new RPMs in nethserver-dev repository and select stable packages ready for production 3. Rebuild each selected package and publish it to nethserver-testing repository 4. Test new RPMs in existing machine and in a new freshly installed one 5. If all test pass, move RPMs to repository nethserver-update 6. Build the new ISO See Building ISO.

1.2.6 New packages Before creating a new package, make sure it’s a good idea. Often a simple documentation page is enough, and it requires much less effort. When trying new things, just take care to write down on a public temporary document (maybe a wiki page) all steps and comments. If the feature collects many requests, it’s time to think about a new package. Otherwise, the temporary document can be moved to a manual page. When creating a new package, make sure the following requirements are met: • Create an issue describing the package • Request the creation of a new repository (including Github mirror) • Add the repository to Redmine to keep track of source changes from issues • Add new record inside the package list http://dev.nethserver.org/projects/nethserver/wiki/Packages • Add a wiki page describing the usage of package, the page should be named like the package itself • Request Redmine administrators to add the package on “NethServer package” custom field • If needed, add the package to a yum group as optional or mandatory package • Add the repository to Ohloh for statics gathering Steps to release a new package 1. Update/commit changelog 2. Add git tag 3. Build RPM 4. Publish the RPM to nethserver-update yum repository

1.2. Development process

7

NethServer Documentation, Release 0

5. Push git tag and package changelog 6. If needed, update yum groups file See Building RPMs.

1.3 RPM package rules 1.3.1 Naming and events conventions Each package name MUST be composed of • a prefix, corresponding to the product name: nethserver-, neth-, .... • the feature/function/daemon/software: base, directory, httpd-admin ... Each package MUST contain: • -update event, raised each time the package is installed/updated and when the system is re-configured (for instance, after another package has been uninstalled) The update event should: • configure the package on first install • take care of upgrading current installation in case of package update Note: You should not add code in %post and %pre sections of the spec file. All the logic must be inside the -update event. Each package MAY contain: • -save event, raised by the console or the web interface to adjust the package configuration after some DB value has changed; • -conf action, to execute package-specific configuration commands. This action MUST be invoked during the -update event. For example, given a package named nethserver-dnsmasq: • update event: nethserver-dnmasq-update • save event: nethserver-dnmasq-save • configuration action: nethserver-dnmasq-conf

1.3.2 Install/Update process Just after a package transaction (install/update), the NethServer yum plugin will: • execute all nethserver-update-: events of installed/updated packages in the current transaction • execute runlevel-adjust event to start/stop all configured services • execute firewall-adjust event to open/close firewall ports In case of manual installation , the update, firewall-adjust and runlevel-adjust events must be fired manually using the signal-event command.

8

Chapter 1. Rules and conventions

NethServer Documentation, Release 0

1.3.3 Uninstall process After a package is removed, the NethServer yum plugin will: • execute all nethserver-update-package event of installed packages • execute runlevel-adjust event to start/stop all configured services • execute firewall-adjust event to open/close firewall ports

1.3.4 Service packages A service package is an RPM which is responsible for a system service configuration and management. The package must follow all rules listed above plus some more conventions. Configuration DB default Mandatory: • type: service • status: the current service status, can be enabled or disabled Optional: • TCPPort: a tcp listening port • UDPPort: a udp listening port • TCPPorts: a list of tcp listen ports. Eg: 123,678 • UDPPorts: a list of udp listen ports. Eg: 123,678 For example, the package nethserver-puppet managing the service puppet will contain: • /etc/e-smith/db/configuration/defaults/puppet/type • /etc/e-smith/db/configuration/defaults/puppet/status Beside this, each packge MUST always declare its own set of keys and properties inside default databases. Events and actions nethserver-base package provides two generic events: • runlevel-adjust: enable/disable the service using chkconfig command and start/stop the service • firewall-adjust: read TCPPort(s) and UDPPort(s) props and open the specified ports in the firewall configuration Both events are handled by the system, so there is no need to link these events into the package. Further documentation: perldoc /usr/share/perl5/vendor_perl/NethServer/Service.pm Orphan services During runlevel-adjust event, the system will stop any orphan service. A orphan service is a running service not controlled by any nethserver-package. A service is an orphan if there is a service record in configuration db, and there is no db defaults (in /etc/e-smith/db/configuration/defaults).

1.3. RPM package rules

9

NethServer Documentation, Release 0

1.4 Internationalization These are the coding conventions for NethServer i18n. Each package repository should respect them. • The developer prepares the translation source strings when he writes the code. • Each translation catalog must be mapped to a resource on Transifex. • Whenever new strings are added or existing ones are changed, source catalogs must be pushed into Transifex with Transifex client: tx push -s

To configure the Transifex client execute the txinit.sh script on the repository root. The script can be executed multiple times, if new catalogs are added to the repository. Currently both gettext and Server Manager specific format is supported for language catalogs.

1.4.1 gettext The xgettext command can be used to extract strings from the source code. The resulting .pot file must be named locale/.pot. gettext example In this example we will translate a new TODO message for the web interface. Steps to setup translation for a new todo: 1. Be sure gettext is installed in your system 2. Add the todo script to the git repository 3. Create locale directory inside the git repository root: mkdir -p locale

4. Extract the strings (from a Python source code):

xgettext --msgid-bugs-address "[email protected]" --package-version "$(git describe --tags) --package-name "$(basename `pwd`)" --foreign-user -d "$(basename `pwd`)" -o "locale/$(basename `pwd`) -L Python root/etc/nethserver/todos.d/*

5. Create the language file from template: mkdir -p locale/en/LC_MESSAGES cp locale/.pot locale/en/LC_MESSAGES/.po

6. Substitute following fields in po file: • DESCRIPTIVE • LANGUAGE • CHARSET (must be UTF-8) • FIRST AUTHOR , YEAR. 7. Add translations. You can try poedit editor

10

Chapter 1. Rules and conventions

NethServer Documentation, Release 0

1.4.2 Server Manager All source language catalog files are placed in root/usr/share/nethesis/NethServer/Language/en. We assume en is en-US. The catalog format is a common PHP file where the array variable $L is assigned some keys: . 2. Use UTF-8 encoding.

Given a module with name “Test”, the source language file will be root/usr/share/nethesis/NethServer/Language/en/ When the Server Manager is running in debug mode, missing translation labels can be found in /var/log/messages. To enable the debug, • Unlock index_dev.php controller: touch /usr/share/nethesis/nethserver-manager/debug

• Prepend index_dev.php on URLs path, eg: https:///index_dev.php/en/.

1.4. Internationalization

11

NethServer Documentation, Release 0

12

Chapter 1. Rules and conventions

CHAPTER 2

Architecture

2.1 Databases 2.1.1 Overview All user-editable configuration parameters on NethServer are stored in plain text database. These values are used to generate the system configuration files, such as those found in the /etc/ directory. The configuration databases may be modified by various programs on the system, including the web interface or scripts run from the command line by a system administrator. Each entry in the database is either a simple key/value pair or a key and a collection of related property/value pairs.

2.1.2 Simple entries Simple configuration database entries take the form of a key/value pair: [root@nsrv -]# config show SystemName SystemName=myserver

2.1.3 Complex entries More complex entries consist of a key, a type, and a collection of property/value pairs: [root@nsrv -]# config show sysconfig sysconfig=configuration Copyright= ProductName=NethServer Registration=none Release=4 Version=6.4

Use complex entries whenever possible

2.1.4 Access from the command line You can access database entries from the command line using the config command, as shown above, or the db command. The config command provides a shorthand for accessing the configuration database. The following commands are equivalent:

13

NethServer Documentation, Release 0

[root@nsrv -]# config show SystemName SystemName=nsrv [root@nsrv -]# db configuration show SystemName SystemName=nsrv

The db command allows you to access all of the databases. For example to show the details of the test entry from accounts db: [root@nsrv -]# db accounts show test test=user City= Company= Department= FirstName=test LastName=test PhoneNumber= Street= Uid=5000 VPNClientAccess=yes VPNRemoteNetmask=255.255.255.0 VPNRemoteNetwork=192.168.1.0

For more options see help of db command: db -h

2.1.5 Access via the Perl API You can also access configuration database entries programmatically using the esmith::ConfigDB and related Perl modules, which are abstractions for the esmith::DB module. For example, we can retrieve and show the admin account details like this: use esmith::AccountsDB; my $db = esmith::AccountsDB->open or die "Couldn't open AccountsDB\n"; my $admin = $db->get("admin") or die "admin account missing from AccountsDB\n"; print $admin->show();

For documentation on Perl API use the perldoc command. Eg: perldoc esmith::ConfigDB

2.1.6 Database initialization The configuration databases are initialized from files in the /etc/e-smith/db/ hierarchy. These files can perform one of three actions: • Create a database entry and set it to a default value, if the entry does not already exist. • Migrate an entry from a previous value to a new value. • Force a database entry to a specific value, regardless of its current setting (use with care!) This design allows each package to provide part of the system configuration, or migrate the system configuration values as required. Note that a single database property can only be owned by one package. Database initialization is run during system install, system upgrade and after new software has been installed.

14

Chapter 2. Architecture

NethServer Documentation, Release 0

If you examine the /etc/e-smith/db/configuration/ directory you will see three subdirectories: defaults/, force/ and migrate/ to match the three options above. A similar structure exists for each of the other databases. A new database can be created by populating a new directory tree under the /etc/e-smith/db/ directory.

Configuration databases can also be initialized using a special /usr/libexec/nethserver/initialize--databa script, where dbname is the database name. For example: /usr/libexec/nethserver/initialize-mycustomdb-databas Defaults files Defaults files are simple text files. If the corresponding database key/property already exists, it is skipped. Otherwise, the key/property is created and the value loaded. For example, this file: [root@nsrv -]# cat /etc/e-smith/db/configuration/defaults/sshd/status enabled

It would create the sshd database entry if it doesn’t already exist, create the status property for that entry, again if it doesn’t already exist, and finally set the status property to enabled. Forcing database initialization

Simply call the action: /etc/e-smith/events/actions/initialize-default-databases Force files Force files are just like defaults files, except they overwrite the existing value. So, this file: [root@nsrv -]# cat /etc/e-smith/db/configuration/force/sysconfig/Version 6

It would create the Version property of the sysconfig entry and unconditionally set its value to 6. Warning: Do not use force fragments if not really necessary!

Migrate fragments Migrate fragments are small pieces of Perl text which can be used to perform more complex migrations than is possible with defaults and force files. They would normally be used to replace database keys or properties with new names, or to adjust policy settings during an upgrade. Each fragment is passed a reference to the current database in the $DB variable. This variable is an instance of the appropriate esmith::DB subclass, e.g. esmith::AccountsDB when the accounts database migrate fragments are being executed. This means that you can use the methods of that subclass, for example esmith::AccountsDB->users(). Here is an example of a migrate fragment, which replaces the outdated popd entry with the new name pop3: { my $popd = $DB->get("popd") or return; my $pop3 = $DB->get("pop3") || $DB->new_record("pop3", { type => "service" }); $pop3->merge_props($popd->props); $popd->delete; }

2.1. Databases

15

NethServer Documentation, Release 0

This fragment checks whether the database (the configuration database in this case) has a popd entry. If that entry does not exist, the migrate fragment returns immediately. If the popd entry exists, we need to convert it, so we retrieve the pop3 entry (or create it if it doesn’t already exist). We then merge the properties from the popd entry into the pop3 entry and finally delete the popd entry. If this migrate fragment is run again, it will return immediately as the popd entry has already been deleted. Important notes about migrate fragments

• Please be careful with migrate fragments. Although they should only modify entries within the current database, there are no restrictions placed on what they can do. The ability to open and even modify other databases may be required to perform a migration. • Migrate fragments must be safe to run multiple times. They should migrate the value when required and do nothing in other cases. • Migrate fragments should never call croak or die. This will cause the database migration to stop. If an error is detected, call carp or warn to note the error in the logs. • Migrate fragments should call good termination with return(0) rather than exit(0). • Migrate fragments should be owned by the package requiring the migration so that the migration only occurs when that package is installed. • Migrate fragments should be self-contained and ideally perform only one migration per fragment. • DO NOT USE to initialize default database values.

2.1.7 Evaluation order When a database is loaded: • migrate scripts are run first • then defaults are loaded • and finally any force files are loaded. This order allows migration of old format entries to occur prior to loading of new default values. Remember, defaults will not change an existing database property.

2.1.8 Best practices • The configuration databases should only be modified using the tools and APIs provided. • The order of the entries and the order of properties is undefined. • The keys and property names are currently treated in a case-sensitive manner, though this may change in the future. Do not create keys or property names which differ only by their case. • Underscores and hyphens are valid in key and property names, but should normally be avoided. • Do not “overload” an existing property with a new value. If the existing values do not meet your requirements, discuss your implementation with the developers. Values which are not known by the base may cause serious issues on upgrade. If the existing panels have three choices, do not invent new choices without enhancing the panel to support them. • The type pseudo-property is used internally and is reserved .

16

Chapter 2. Architecture

NethServer Documentation, Release 0

• By convention, database keys are lower case, and property names are stored in mixed case. The type, status and access properties are exceptions to this convention. • The storage location and internals of the databases is subject to change. • The configuration databases are currently /var/lib/nethserver/db/ directory.

stored

as

pipe-delimited

flat

text

files

in

the

Namespace issues All entries in a single database share the same namespace. Users, groups, information bays, printers, and other entries in the accounts database currently all share one namespace. This means that you cannot have a user with the same name as an information bay, group or other entry in the accounts database. However, it would be possible to have a host named fredfrog as well as a user named fredfrog as they are stored in separate databases and thus different namespaces.

2.1.9 List of available database Table of databases The following table summarizes • the database name • the perl module that manages it and • the package that provides it Databases provides by the base system: Name configuration hosts networks domains

Perl module esmith::ConfigDB esmith::HostsDB esmith::NetworksDB esmith::DomainsDB

Package nethserver-base nethserver-hosts nethserver-base nethserver-mail-common

Description

Each modules can define its own new databases. Some relevant databases are: Name accounts domains

Perl module esmith::AccountsDB esmith::DomainsDB

Package nethserver-directory nethserver-mail-common

Description

2.2 Templates 2.2.1 Design of the template system Every piece of software has its own configuration format, and writing parsers for each one is a complex, timeconsuming and error-prone process. The template system software avoids the whole issue by using templates which generate the correct configuration. Configuration files are over-written when templates are expanded. In a few specific cases, the existing configuration file is parsed and rewritten in-place. This is done where the configuration file (e.g. /etc/fstab) is also automatically updated by some other process.

2.2. Templates

17

NethServer Documentation, Release 0

Templates are stored under /etc/e-smith/templates/ in a directory hierarchy which matches the standard filesystem. For example, the template for /etc/inittab is stored in the /etc/e-smith/templates/etc/inittab/ directory. Each template is stored as a directory of template fragments and processed by the Perl Text::Template module. The template fragments are concatenated together in ASCIIbetical order (US-ASCII sort order) and the complete file is parsed to generate the appropriate configuration files for the service. The use of fragments is part of NethServer modular and extensible architecture; it allows third-party modules to add fragments to the configuration where necessary.

2.2.2 The Text::Template module The Text::Template module allows arbitary Perl code to be embedded in a template file by surrounding it in curly braces ({ and }). The code inside the braces is interpreted and its return value replaces the section between, and including, the braces. For instance: The answer is { 40 + 2 }

becomes: The answer is 42

Variables can be passed in from the program which is expanding the template, hence: { $OUT = ';' for my $item ( qw(bread milk bananas) ) { $OUT .= "\* $item\n"; } }

would expand to: Shopping list * bread * milk * bananas

The template system uses this mechanism to automatically pass in global configuration variables from the configuration database which can then be used to fill out the configuration files. For example, the /etc/hosts template could be fairly simple and composed of two fragments: [root@test hosts]$ ls /etc/e-smith/templates/etc/hosts 10localhost 20hostname

Fragments can have static content. For example: 127.0.0.1

localhost

The second is more complex and relies on values from the configuration database: { $OUT .= "$LocalIP\t"; $OUT .= " ${SystemName}.${DomainName}"; $OUT .= " ${SystemName}"; }

18

Chapter 2. Architecture

NethServer Documentation, Release 0

Note that the whole fragment is enclosed in braces. Within those braces is a section of Perl code. When this template is expanded, it results in the following configuration file: # ================= DO NOT MODIFY THIS FILE ================= # # Manual changes will be lost when this file is regenerated. # # Please read the developer's guide, which is available # at https://dev.nethesis.it/projects/nethserver/wiki/NethServer # original work from http://www.contribs.org/development/ # # Copyright (C) 2015 Nethesis S.r.l. # http://www.nethesis.it - [email protected] # 127.0.0.1 localhost 192.168.10.1 nsrv.nethesis.it nsrv

The header block comes “for free” as part of the template system, courtesy of an optional file template-begin, which is always processed as the first fragment. If it isn’t provided, the text shown with # comments is included. If target configuration file do not support line comment beginning with #, please provide a custom or empty template-begin. The other lines are provided by the two fragments shown above. Note the use of the configuration database variables: $LocalIP, $SystemName and $DomainName. All simple entries in the configuration database are provided as global variables to the templates. Note that all of the template fragments are concatenated together before evaluation, so it is possible to set values in fragments which are used in later fragments. This is a very useful model for reducing the code in individual template fragments. The complex entries in the configuration database are also provided as global variables to the templates. However, they are provided as Perl hashes instead of simple scalars. For example, here is how you might configure the Network Time Protocol (NTP) server /etc/ntp.conf file: server { $ntpd{NTPServer} } driftfile /etc/ntp/drift authenticate no

The NTPServer setting is stored in the ntpd configuration database record, and so can be accessed via the hash accessor $ntpd{NTPServer}. template-begin and template-end Each template directory can contain two optional files template-begin and template-end . The templatebegin file is always processed as the first file of the template, and the template-end file is always processed as the last file. If the directory does not contain a template-begin file, /etc/e-smith/templates-default/template-begin is used automatically.

the

contents

of

If the directory does not contain a template-end , nothing is appended to the template output. It is mostly used to provide the closing block for configuration files written in languages such as HTML and PHP, through a link to an entry in the templates-default/ directory.

2.2. Templates

19

NethServer Documentation, Release 0

/etc/e-smith/templates-default The /etc/e-smith/templates-default directory contains a set of template-begin and template-end files for various languages. For example, if your template generates a perl script, you would link template-begin to /etc/e-smith/templates-default/template-begin-perl and automatically get the #!/usr/bin/perl -w line and a comment containing the contents of the default template-begin file. Note: You may also need a templates.metadata configuration file if your generated file needs to be executable.

Template fragment ordering Template fragments are assembled in ASCII-betical order, with two exceptions: template-begin always comes first, and template-end always comes last. Template fragments are often named to start with a two digit number to make the ordering obvious, but this is not required. Templates for user home directories: templates-user Most of the templates on the system map to single, fixed output files, such as /etc/hosts. However, templates are also used to generate configuration files such as mail delivery instructions for users. These templates are stored in the /etc/e-smith/template-user/ tree. As these templates have a variable output filename, they are expanded using small pieces of Perl code in action scripts. Local site overrides: templates-custom and templates-user-custom It is possible that the standard templates are not correct for a particular installation, and so the local system administrator can override the existing templates by placing files in the templates-custom tree. This is a parallel tree to the normal templates hierarchy, and is normally empty. There is also a template-user-custom tree for overriding entries in the templates-user tree. If a templates-custom entry exists for a template, it is merged with the standard templates directory during template expansion, using the following rules: • If a fragment of the same name exists in both templates and templates-custom, the one from templates-custom is used, and the one from the standard templates tree is ignored. • If the fragments in templates-custom have different names from those in templates, they are merged into the template as if they were in the templates directory. • If the templates-custom entry is a file, rather than a directory, it completely overrides the standard template. To make this concrete, let’s assume we /etc/e-smith/templates/etc/book.conf:

have

the

following

template

structure

in

10intro 30chapter3 40chapter4 80synopsis

and in /etc/e-smith/templates-custom/etc/book.conf: 30chapter3 50chapter5

The resulting template would be processed in this order: 20

Chapter 2. Architecture

NethServer Documentation, Release 0

• template-begin from /etc/e-smith/templates-default • 10intro from /etc/e-smith/templates/etc/book.conf • 30chapter3 from /etc/e-smith/templates-custom/etc/book.conf • 40chapter4 from /etc/e-smith/templates/etc/book.conf • 50chapter5 from /etc/e-smith/templates-custom/etc/book.conf • 80synopsis from /etc/e-smith/templates/etc/book.conf • template-end (empty), nominally from /etc/e-smith/templates-default How to resolve conflicts with standard templates

It is possible that the standard templates may specify behaviour which is not appropriate for your application. In many cases the templates will be driven by configuration database settings which allow their behaviour to be customized, which should be the first thing to check. In many cases, your application only needs to extend the behaviour of the template by adding one or more fragments. This should be your second option and can be achieved by simply adding your fragment in the correct place in the list of fragments. In rare cases the standard template specifies a behaviour which conflicts with your application. In these cases, you should do all of the following: • Create a templates-custom directory to match the existing one in the templates hierachy. • Copy the conflicting fragment, and only that fragment, to the templates-custom directory. The fragment should have the same name in both directories. At this point you have not changed the behaviour of the system as the templates-custom entry will be preferred, but will behave identically. • Modify the copy in templates-custom to suit your required behaviour. • Inform the NethServer team about the problem. Please attach your modified template (or even better, a patch file) and provide details of why you think that the standard template should be changed. The expansion of templates To expand your custom templates to their destination you have to use the following command: expand-template where template destination has to be changed with the true path to the configuration file. For Example you want to add something to the samba configuration, then you have to build a custom template fragment under /etc/e-smith/template-custom/etc/samba/smb.conf/ directory and execute the command: expand-template /etc/samba/smb.conf Subdirectory templates It is also possible to split templates into further subdirectories. This can be very useful for evaluating the same fragments in a loop, for example for each virtual domain in httpd.conf or each ibay in smb.conf. Two examples of this can be found in /etc/e-smith/templates/etc/httpd/conf/httpd.conf/80VirtualHosts which loops over the /etc/e-smith/templates/etc/httpd/conf/httpd.conf/VirtualHosts/ directory, and /etc/e-smith/templates/etc/smb.conf/90ibays which performs a similar loop over the /etc/e-smith/templates/etc/smb.conf/ibays/ directory.

2.2. Templates

21

NethServer Documentation, Release 0

2.2.3 Template expansion The system is designed to ensure consistent and reliable operation, without requiring command-line access. Whenever an event is signalled, the relevant templates for that event are expanded and the services are notified of the configuration changes. Requesting expansion of a template in an event is a simple matter of creating an empty file under the templates2expand hierarchy for that event. See events manual chapter for further information.

2.2.4 Template permissions and ownership: templates.metadata Templates are normally expanded to be owned by root and are not executable, which is a reasonable default for most configuration files. However, templates may need to generate configuration files which are owned by a different user, or which need to be executable or have other special permissions. This can be done by creating a templates.metadata file which defines the additional attributes for the expansion. Note: Configuration files should generally not be writable by any user other than root. In particular, configuration files should not normally be writable by the www user as this poses a significant security risk. Installation advice which says chmod 777 is almost invariably wrong. For example, here is the metadata file /etc/e-smith/templates.metadata/etc/ppp/ip-up.local: UID="root" GID="daemon" PERMS=0755

which sets the group to daemon and makes the script executable. Note that the file is readable by members of the daemon group, but it is not writable by anyone but root. It is also possible to use the same template to generate multiple output files, such as in this example: TEMPLATE_PATH="/etc/sysconfig/network-scripts/route-ethX" OUTPUT_FILENAME="/etc/sysconfig/network-scripts/route-eth1" MORE_DATA={ THIS_DEVICE => "eth1" } FILTER=sub { $_[0] =~ /^#/ ? '' : $_[0] } # Remove comments

The templates.metadata file for route-eth0 just uses eth0 instead of eth1 on the second and third lines. Note also the FILTER setting which allows post-processing of the generated template. There are many examples under /etc/e-smith/templates.metadata/ and the full list of options can be seen with: perldoc esmith::templates

2.2.5 Perl API: processTemplate In rare circumstances you may need to call processTemplate directly. Explicit calls to processTemplate are typically only used when the output filename is variable: use esmith::templates; foreach my $name (@names) { [...] processTemplate( {

22

Chapter 2. Architecture

NethServer Documentation, Release 0

TEMPLATE_PATH => "/etc/myservice/user.conf", OUTPUT_FILENAME => "/etc/myservice/$name.conf" ); [...] }

bq. Content is available under GNU Free Documentation License 1.3 unless otherwise noted. Original document from: http://wiki.contribs.org/

2.3 Actions and events 2.3.1 Actions An action is a program, frequently written in a scripting language, which performs a single task. It is typically an encapsulation of a task usually done by a system administrator, such as editing a configuration file or reconfiguring a service. Actions are not called directly; they are always called by signalling an event. The actions are stored in the /etc/e-smith/events/actions/ directory. These actions are then linked into the relevant events as the same action may need to be performed in more than one event. To create a new action called myaction you simply create a program to perform the action myaction and save it as /etc/e-smith/events/actions/myaction . Actions can be written in any programming language, although additional platform support is provided for Perl code. An example action script is: #!/bin/bash /usr/sbin/lokkit --update

Action script parameters Action scripts are always called with at least one parameter; the name of the current event. Many action scripts can be called with a single additional parameter. This parameter is usually a configuration database key, for example the username being modified. Action scripts rarely require more than two parameters. The details should be stored in the configuration database(s) and only the key should be passed to the action scripts. All configuration details must be stored in the configuration databases and the database key passed as the parameter to the action. This allows other scripts to be added to the event. Since the system passes the name of the current event as the first parameter, it is often beneficial to write action scripts which are polymorphic based on the event name. For example, the code to create a user and the code to modify an existing user may be only slightly different and may well benefit from being in a single script. Example: use strict; my $event = $ARGV[0]; my $myarg = $ARGV[1]; exit 0;

Note: Whenever possible, avoid to call events from within action scripts.

2.3. Actions and events

23

NethServer Documentation, Release 0

Action code libraries To promote code reusability and components abstraction some Perl ules are available under /usr/share/perl5/vendor_perl/NethServer/ /usr/share/perl5/vendor_perl/esmith/. For instance,

modand

NethServer::Password Secret generation and persistent storage, under /var/lib/nethserver/secrets/. NethServer::Service Service manager agnostic API. No matter if a service is managed by systemd, Upstart or SysV init script: use this API to gain control over it. NethServer::Directory Access to LDAP, service accounts and ACL management, low-level user and group management. NethServer::MailServer Obtain accounts and mail addresses relations. Manage IMAP ACLs. esmith::templates Template processing and expansion. esmith::events Event execution and tracking. For more informations about a specific module, refer its perldoc documentation.

2.3.2 Events Events are a mechanism which allows the system to trigger a set of actions in response to actual events that happen on the system. When one of the users interfaces modifies the configuration databases, it must signal an event to regenerate the various server application configuration files according to the new configuration. Note: The user interface must never modify configuration files directly. Neither should to the administrator from command line. Each event is associated with a list of actions which should be performed when the event occurs and is defined as a subdirectory of /etc/e-smith/events/ containing symbolic links to the appropriate actions, loosely modelled after the ‘’System V init’‘mechanism for starting servers. For example, if you examine the /etc/e-smith/events/interface-update directory:

[root@nsrv actions]# ll /etc/e-smith/events/interface-update/ total 8 lrwxrwxrwx. 1 root root 34 Feb 6 11:19 S04interface-config-adjust -> ../actions/interface-config-a lrwxrwxrwx. 1 root root 33 Feb 6 11:19 S25interface-config-reset -> ../actions/interface-config-re lrwxrwxrwx. 1 root root 33 Feb 6 11:19 S30interface-config-write -> ../actions/interface-config-wr lrwxrwxrwx. 1 root root 35 Feb 6 11:19 S70interface-config-restart -> ../actions/interface-configlrwxrwxrwx. 1 root root 36 Feb 6 11:19 S75interface-config-hostname -> ../actions/interface-config drwxr-xr-x. 2 root root 4096 Feb 6 11:20 services2adjust drwxr-xr-x. 3 root root 4096 Dec 18 11:17 templates2expand

The symbolic links are given prefixes such as S15, S85, etc. to specify the order in which the actions should be executed in a similar manner to the System V init mechanism. You can change the actions performed by an event by changing the links in the event directory. You can also create a new event by creating another subdirectory of /etc/e-smith/events/. Implicit actions Most events contain two common tasks: expanding various templates and adjusting (e.g. restarting) the relevant services. For this reason, two implicit actions are included in all events. These implicit actions mean that additional code does not need to be written to perform these common tasks. The implicit actions are represented by entries in the services2adjust/ and templates2expand/ subdirectories. 24

Chapter 2. Architecture

NethServer Documentation, Release 0

services2adjust

The services2adjust/ directory contains links mapping a specific service to the action to perform on that service. For example, if signalling the event in question requires that the ntpd service is restarted, you simply include the link ntpd -> restart in the services2adjust directory. The implicit action services2adjust would then restart the ntpd service. As an example, the services2adjust/ directory for the nethserver-httpd-update event is shown below: # ls> l /etc/e-smith/events/nethserver-httpd-update/services2adjust/ total 0 lrwxrwxrwx. 1 root root 7 Oct 2 09:05 httpd -> restart

templates2expand

The templates2expand/ directory contains a list of the configuration files which need to be regenerated from their templates. This list consists of a collection of empty files with the same file name as the configuration file to be expanded and in a heirarchy mirroring their location on the system. For example, to expand templates for the /etc/samba/smb.conf configuration file, simply include the empty file etc/samba/smb.conf in the templates2expand/ directory of the relevant event. Order of implicit actions

The implicit actions are implemented by inserting the action script generic_template_expand early in the list of actions to be run in an event and the adjust-services action near the end of the list. You should normally link your action scripts in the range S10 to S80 so that they occur after templates2expand and before services2adjust. Note: The generic_template_expand action is currently run at S05 and adjust-services is run at S90.

Signalling events The signal-event program takes an event name as an argument, and executes all of the actions in that event, providing the event name as the first parameter and directing all output to the system log. It works by listing the entries in the event directory and executing them in sequence. So for example, the command: signal-event interface-update

will perform all the actions associated with the interface-update event, which is defined by the contents of the /etc/e-smith/events/interface-update/ directory. Events with arguments

So far we have described the following general principle throughout the system; changes are made by altering the database values, then signalling events. The actions triggered by each event typically regenerate entire configuration files, taking into account the latest configuration information. However, some changes are best made incrementally. For example, consider the user-create event. One of its actions updates the LDAP directory, which it could do by deleting all of the users and recreating them based on the updated accounts database. However, this is inefficient and would lose any additional LDAP attributes which may have been stored. It would be better to simply add the new user incrementally, using the default LDAP schema.

2.3. Actions and events

25

NethServer Documentation, Release 0

But how is the action code to know which user was just added? The new username is passed as an argument to the usercreate event. This way the action programs triggered by the user-create event have a choice. They can either ignore the username argument and regenerate their output based on the updated list of accounts, or they can pay attention to the username argument, retrieve the rest of the information about the new user from the accounts database, and perform the incremental work to add the user. Note: Action scripts should normally take at most two arguments. The first is always the event name. The second optional argument is a key into one of the databases. Events are not function calls. Events are not currently serialized. In most cases overlapping events will not cause issues, but caution should be exercised when events are signalled from programs. Standard events and their arguments The table below summarises the key NethServer events and their argument if required. Remember, each action script is always called with the event name as the first argument. The arguments listed in this table are provided as the second argument. Event certificate-update fstab-update group-create group-delete group-modify group-create-pseudonyms host-create host-delete host-modify hostname-modify ibay-create ibay-delete ibay-modify interface-update logrotate-update trusted-networks-update migration-import password-expired password-modify password-policy-update post-backup-config post-backup-data post-restore-config post-restore-data pre-backup-config pre-backup-data pre-restore-config pre-restore-data pseudonym-create pseudonym-delete pseudonym-modify

26

Arguments

Group key Group key Group key Host key Host key Host key Shared folder key Shared folder key Shared folder key

Path to migration directory Username, expire date User key User key

Pseudonym key Pseudonym key Pseudonym key

Description The server public key certificate has been updated Update /etc/fstab according fo fstab key and remount all fileystems Called when a group is created Called when a group is deleted Called when a group is modified Signalled when the automatic creation of group email address is required Called when a host is created Called when a host is deleted Called when a host is modified Called when the SystemName or DomainName keys have been modified Called when a shared folder is created Called when a shared folder is deleted Called when a shared folder is modified Called when a network interface configuration is updated in networks db Change default log retention and rotation policies The set of trusted networks is changed Import migration data from the given directory The given username password will expire on expiredate Called when a user password is modified Called when the system password policy has been changed Called after configuration backup end Called after data backup end Called after restore of configuration Called after restore of data The pre-backup-config event creates consistent system state for the backup The pre-backup-data event creates consistent system state for the backup Called before restore of configuration Called before restore of data Called when a pseudonym is created Called when a pseudonym is deleted Called when a pseudonym is modified Continued on next page

Chapter 2. Architecture

NethServer Documentation, Release 0

Event user-create user-delete user-modify user-create-pseudonyms user-lock user-unlock system-initialization

Arguments User key User key User key User key User key User key

Table 2.1 – continued from previous page Description Called when a user is created Called when a user is deleted Called when a user is modified Called when the automatic creation of user’s email address(es) is required Called when a user account is locked Called when a user account is unlocked Initialize all system after installation

Handling deletions

When adding a user, the user is created in the accounts database, and various actions, such as creating the Linux account, are performed in the user-create event. However, when deleting a user, we want to maintain the accounts database entry for as long as possible, in case there is information which the actions in the user-delete event might need in order to cleanly delete the users. The system convention for handling deletions is: • Change the type of the entry to mark it as being in the process of being deleted e.g. a’‘user’‘entry becomes a’‘user-deleted’‘entry. • Signal the relevant deletion event - e.g.’‘user-delete’‘ • Remove the entry from the database, but only if the event succeeds. With this approach, the action scripts can decide whether to ignore the’‘user-deleted” entries when performing their tasks. Event logs Warning: Output of event logs will be soon refactored! All events, and all actions run by the event, are logged to the messages system log. Here is an example action log, which has been formatted onto multiple lines to enhance readability: Feb 2 13:22:33 gsxdev1 esmith::event[4525]: S65sshd-conf=action| Event|remoteaccess-update| Action|S65sshd-conf| Start|1138846952 730480| End|1138846953 66768| Elapsed|0.336288

From this single log, we can see the action script name, which event it was called in, when it started, ended and how long it took (0.34 seconds). Now, let’s add an action script which always fails and signal the event again: Feb 2 16:11:54 gsxdev1 esmith::event[4787]: S99false=action| Event|remoteaccess-update| Action|S99false| Start|1138857114 58910| End|1138857114 81920| Elapsed|0.02301| Status|256

Note that this log has a new field Status, which is added if the action script returns a false (non-zero) exit status. Suppressing the Status field when it is zero (success) makes it much easier to find failed actions in the logs.

2.3. Actions and events

27

NethServer Documentation, Release 0

Warning: If an action script fails, the entire event fails. The other actions scripts in the event are run, but the whole event is marked as having failed.

System validators System validators provide an extensible UI-independent data validation layer. On one hand UI implements fast grammar and/or syntax checks on input data. On the other, the system validators performs in-depth system consistency checks. Design

Validators have a behaviour very similar to events. • A validator is a directory inside /etc/e-smith/validators. • Each validator directory has a descriptive name, eg. user-name for a validator which validate a new user name. • A validator is composed by an arbitrary number of actions saved /etc/e-smith/validators/actions directory and linked inside validator directory.

inside

• A success validation occurs when all scripts return 0 (success validation) or at least one script returns 2 (sufficient valid condition). A validator action are always called with a single parameter which is the value to be validated. Actions must return one of these exit values: • 0: successful validation • 1: validation failed • 2: sufficient validation • other value: specific error state When a script returns 2 (sufficient validation) no further script will be processed. Inside nethserver-devtools package there is validator_actions() function which help creating links to actions just like event_actions function. See perldoc esmith::Build::CreateLinks for details. Invoking a validator: validate

Eg: validate user-name goofy

h2. Implementation See #303.

2.4 Services A service is a software which usually runs in background. The system will ensure service status accordingly to its configuration. A service in configuration database is something like this:

28

Chapter 2. Architecture

NethServer Documentation, Release 0

httpd=service status=enabled access=public TCPPorts=80,443

Where httpd is the service name and status tells the system if the service should be enabled or disabled. When the status property is switched between enabled/disabled state, the change will be reflected into runlevel configuration using chkconfig command. If both Upstart and SysV scripts are present, Upstart has the precedence and SysV script is disabled. For example see httpd-admin service. This is what runlevel-adjust event and action do for all configured services. There is also another action called adjust-services which does the same thing for services registered on a single event. A service without a record in the configuration database is ignored and can be manually manged using chkconfig and service commands. See Add a new service.

2.4.1 Control a service Enable a service: config setprop myservice status enabled signal-event runlevel-adjust

Disable a service: config setprop myservice status disabled signal-event runlevel-adjust

Where myservice is the service name to be enabled or disabled.

2.4.2 Access network service A network service is a service running on the server which expose UDP or TCP ports. Ports can be listed in following properties: • TCPPort: a single TCP port • TCPPorts: a comma separated list of TCP ports • UDPPort: a single UDP port • UDPPorts: a comma separated list of UDP ports If both TCPPort and TCPPorts properties are set, TCPPorts has the precedence. If both UDPPort and TCPPorts properties are set, UDPPorts has the precedence. A service can be accessible from public or private LAN. This configuration is saved on access property. The property can have one of the following values: • none: the service is accessible only from localhost, no port is open • private: the service is accessible only from green interfaces • public: the service is accessible from green and red interfaces, but no blue and orange Example of a service with UDP port 1122 open to the Internet: config setprop myservice status enabled UDPPort 1122 access public

2.4. Services

29

NethServer Documentation, Release 0

Example of a service with TCP ports 1122 an 2233 open to local network: config setprop myservice status enabled TCPPorts 1122,2233 access private

The ports are opened only if the status property is set to enabled. Custom access Each network service can have one or both of following properties: • AllowHosts: listed hosts can always access the service • DenyHosts: listed hosts can never access the service Both properties can be a list of IPs or CIDR networks and are honored only if access is seto to private or public

2.4.3 Add a new service Any software can configure the init system using the standard chkconfig command. This approach always work for third-party software. On the other hand, if the service must be controlled by NethServer, create a new record inside configuration database: config set myservice service status enabled

Where myservice is the name of the new service. Make sure also there are defaults values inside the directory /etc/e-smith/db/configuration/defaults: if the key is present inside the configuration database, but not inside defaults, the service will be stopped. Given the above example, create these files: mkdir -p /etc/e-smith/db/configuration/defaults/myservice echo "service" > /etc/e-smith/db/configuration/defaults/myservice/type echo "enabled" > /etc/e-smith/db/configuration/defaults/myservice/status

Signal the new service to the system: signal-event runlevel-adjust

2.4.4 Add a new network service If a service not controlled by NethServer needs one or more open ports, use the TCPPort(s) or UDPPort(s) prop to declare the port(s) and signal the firewall to open it: config set fw_myservice service status enabled TCPPort 12345 access private signal-event firewall-adjust

Otherwise, if the service is controlled by NethServer, you can add the properties directly to the service key. For the service myservice on above example: config set myservice service status enabled TCPPort 12345 access private signal-event firewall-adjust

See Firewall and Gateway.

30

Chapter 2. Architecture

CHAPTER 3

Implementation

3.1 Filesystems options Some programs may need special filesystem options to work correctly. For example, Samba needs acl and user_xattr to enable fully compatibility with Windows systems. NethServer add a special fstab key inside the configuration e-smith db. Each prop of fstab is in the form mountpoint=options. For example: fstab=configuration /=defaults,acl /boot=defaults

The entries are not mandatory, if a filesystem hasn’t an associated property, no modification will be done. After each modification to fstab properties, the fstab-update event must be fired. The fstab-update event will expand the /etc/fstab file and then remount all filesystems except for types: swap proc sysfs devpts. Example: config setprop fstab / defaults,acl signal-event fstab-update

3.2 DNS The system will resolve host and domain names using DNS queries to external DNS servers. The configuration is saved inside the dns key from nethserver-base package. Properties: • NameServers: comma separated IP list of external DNS • role: can be set to none or resolver. If role is set to none the server will always use external DNS. For resolver role see DNS server. Example: dns=configuration NameServers=8.8.8.8,208.67.222.222 role=none

31

NethServer Documentation, Release 0

3.2.1 Hosts The system can handle local DNS records. When the server performs a DNS lookup, first it will search inside local DNS records. If no local record is found, an external DNS query will be done. Note: Local DNS record will always override records from external DNS servers. DNS records are called hosts and are saved inside the hosts database. Each entry is saved inside the /etc/hosts file. There are three types of records: • local: hosts inside the internal network • remote: hosts outside the internal network • self: alias for the server itself Records of type local and remote can have following properties: • IpAddress: address of the host • Description: optional description • MacAddress: mac address of the host. Used only for DHCP reservation. See IP reservation. For hosts inside local network, the record key doesn’t have the domain part. Example: host1=local Description=Internal network host #1 IpAddress=192.168.1.23

For hosts outside local network, the record key must have the domain part. Example: external.otherdomain.tld=remote Description=Other domain host IpAddress=8.9.10.11

Records of type self can have following properties: • Description: optional description Example: vhost1.domain.tld=self Description=Virtual Host #1

3.2.2 DNS server The system uses dnsmasq as DNS and DHCP server and it directly resolves all hosts inside its domain. All other names will be queried to external DNS servers. The server will forward reverse lookups to upstream DNS servers, only if upstream DNS servers are inside a private network (eg. network address is 192.168.x.x). The option bind-interfaces is always enabled, as consequence (from dnsmasq man): This option has been patched to always use SO_BINDTODEVICE socket option when binding to interfaces. As consequence, dnsmasq WILL NOT ANSWER to any DNS Queries that come to the socket with the correct destination IP address, but originally on different interface. This behavior differs from the original dnsmasq upstream version and is used for security reasons. 32

Chapter 3. Implementation

NethServer Documentation, Release 0

Properties: • CacheSize: entry to be cached by server, default is 4000 • dhcp-boot: directly pass parameters to dhcp-boot option • except-interface: comma-separated list of interfaces. Do not listen to listed interfaces, useful to avoid conflicts with libvirt • tftp-status: can be enabled or disabled. If enabled, enable the TFTP server for BOOTP (port 67) • access: default is private, do NOT set to public Database example: dnsmasq=service AllowHosts= CacheSize=4000 DenyHosts= TCPPort=53 UDPPorts=53,67 access=private dhcp-boot=pxelinux.0,myserver.mydomain.com,192.168.1.1 except-interface=virbr0,tunspot status=enabled tftp-status=enabled

3.3 DHCP Warning: This page is outdated since nethserver-dnmasq-1.3.0 release The system can act as DHCP server for the local network. Machines which are configured by DHCP have their names automatically included in the DNS server. The DHCP can be enabled only on green and blue interfaces (see Roles and zones). Configuration is saved inside the dhcp database. Each record of range type is associated to an ethernet interface and can have following properties: • status: can be enabled or disabled • DhcpRangeStart: first IP address of DHCP range • DhcpRangeEnd: last IP address of DHCP range • DhcpLeaseTime: seconds of lease time. Default is 86400 • DhcpGatewayIp: (optional) set a custom gateway ip. If not set, the gateway is the ip address of associated interface (record key) The key of the record is the name of the associated interface. Example: eth0=range DhcpGatewayIp= DhcpLeaseTime=86400 DhcpRangeEnd=192.168.1.100 DhcpRangeStart=192.168.5.200 status=enabled

3.3. DHCP

33

NethServer Documentation, Release 0

Hosts inside the blue network can always access the local DNS server. Note: If a record is a related to an interface without a role, the record is automatically deleted. The gateway for clients will be: • if set, the value of property DhcpGatewayIp • otherwise if the server has a red interface, the gateway is the IP address of the interface where the DHCP is enabled (eg. IP of the blue interface for clients in the guest’s network) • otherwise if the server has only a green interface, the gateway of the green interface will be used

3.3.1 IP reservation It’s possible to reserve IPs for specific devices associating the MAC address of the device with the reserved IP. The reservation is saved inside the hosts database. Example: host1=local Description=Internal network host #1 IpAddress=192.168.1.23 MacAddress=08:00:27:48:BF:F3

3.4 Log retention and rotation By default logs are rotated weekly and kept for 4 weeks. Some packages come with different defaults, but the majority do not specify a custom rotate value. Logrotate db property: • Rotate: rotation frequency, can be daily, weekly, monthly. Default is weekly • Times: rotate log files Times number of times (days, weeks or months) before being removes, default is 4 • Compression: can be enabled or disabled. Defaults is disabled Example: logrotate=configuration Compression=disabled Rotate=weekly Times=4

Keep logs for 6 months, rotate once a week: config setprop logrotate Rotate weekly config setprop logrotate Times 24 signal-event logrotate-update

3.5 Network Network configuration is saved inside the NetworksDB (/var/lib/nethserver/db/networks).

34

Chapter 3. Implementation

NethServer Documentation, Release 0

Example of a database containing an interface: eth0=ethernet bootproto=none device=eth0 gateway=192.168.1.254 hwaddr=xx:yy:18:da:dd:01 ipaddr=192.168.1.1 netmask=255.255.255.0 onboot=yes role=green

Each entry describes a network interface according to CentOS/RHEL specification for network-scripts files: = type role = =

The type variable is the type of interface. Valid values are: • ethernet • bond • bridge • alias • ipsec • xdsl The variable is the name for the device. The role property is a mandatory parameter which describes the interface role. Valid values are: • green • orange • blue • red If the role property is empty, the interface is not used by the system. There are also 4 special roles: • bridged: interface is part of a bridge • slave: interface is part of a bond • alias: interface is an alias of another interface • xdsl-disabled: xdsl disabled interface See also Roles and zones for the meaning of each color. All / are all valid CentOS network parameter for the specified interface. All parameters must be lowercase. Example: • ippaddr • hwaddr • dhcp_hostname • netmask

3.5. Network

35

NethServer Documentation, Release 0

• slave • ... All parameters will be mapped 1-to-1 to the configuration file Example One green ethernet:

db networks set eth0 ethernet role green hwaddr xx:yy:27:DE:B6:51 ipaddr 192.168.1.4 netmask 255.255.

File content:

green=ethernet|bootproto|static|device|green|hwaddr|xx:yy:27:DE:B6:51|ipaddr|192.168.1.4|netmask|255.

3.5.1 Bond options Any property starting with BondOpt prefix is used as bonding options. Example: bond0=bond BondOptMode=0 BondOptMiimon=80 bootproto=none gateway=192.168.1.100 ipaddr=192.168.1.2 netmask=255.255.255.0 role=green

Templates The network database can be manipulated using the esmith::NetworksDB perl module. For more information use: perldoc esmith::NetworksDB

If you need to access the local ip address within a template, use this code snippet: use esmith::NetworksDB; my $ndb = esmith::NetworksDB->open_ro() || return;; my $LocalIP = $ndb->green()->prop('ipaddr') || '';

Note: Old templates used a variable called LocalIP to access the green ipaddress. This variable is no more available.

Events All network configurations are applied by interface-update event. Database initialization All interfaces are imported from configuration /usr/libexec/nethserver/update-networks-db .

files

to

database

using

the

script:

The networks database is updated Whenever an interface is plugged into the system. 36

Chapter 3. Implementation

NethServer Documentation, Release 0

Best practices

3.5.2 DHCP on red interfaces When configuring a red interface in DHCP mode, enable also the above options: • peer_dns to avoid resolv.conf overwriting from dhclient • persistent_dhclient to enforce dhclient to retry in case of lease request errors Remember also to remove all gateway ip address from green devices. This configuration will create the correct routes and correctly set DHCP options on dnsmasq.

3.5.3 Bridge Create a bridge interface from command line. The new interface will have green role (eth0 was the previous green interface):

db networks delprop eth0 ipaddr netmask bootproto db networks setprop eth0 role bridged bridge br0 db networks set br0 bridge bootproto static device br0 ipaddr 192.168.1.254 netmask 255.255.255.0 onb signal-event interface-update

3.5.4 Reset network configuration In case of misconfiguration, it’s possible to reset network configuration by following these steps. 1. Delete all logical and physical interfaces from the db Display current configuration: db networks show

Delete all interfaces: db networks delete eth0

Repeat the operation for all interfaces including bridges, bonds and vlans. 2. Disable interfaces Physical interfaces: ifconfig eth0 down

In case of a bridge: ifconfig br0 down brctl delbr br0

In case of a bond (eth0 is enslaved to bond0): ifenslave -d bond0 eth0 rmmod bonding

3. Remove configuration files Network configuration files are inside the /etc/sysconfig/network-scripts/ directory in the form: /etc/sysconfig/network-scripts/ifcfg-. Where devicename is the name of the interface like eth0, br0, bond0. 3.5. Network

37

NethServer Documentation, Release 0

Delete the files: rm -f /etc/sysconfig/network-scripts/ifcfg-eth0

Repeat the operation for all interfaces including bridges, bonds and vlans. 4. Restart the network After restarting the network you should see only the loopback interface: service network restart

Use ifconfig command to check the network status. 5. Manually reconfigure the network Choose an IP to assign to an interface, for example 192.168.1.100: ifconfig eth0 192.168.1.100

Then reconfigure the system: signal-event system-init

The interface will have the chosen IP address. 6. Open the web interface and reconfigure accordingly to your needs

3.6 Auto-generated random URL Sometimes you need to install web applications which don’t have built-in authentication. A good solution can be an auto-generated random URL known only to some special users. It’s also a best practice to restrict access to those applications using Apache allow and deny rules. This feature is implemented in nethserver-lib using genRandomHash function. The function will generate a SHA1 random hash Example from [[nethserver-collectd-web]]: my $alias = $collectd->prop('alias') || ""; # initialize alias if needed if ($alias eq "") { $alias = esmith::util::genRandomHash(); $confdb->set_prop('collectd-web','alias',$alias); }

Random alias should be generated inside an action, like -conf. The action must be executed before template-expand in a position before 05. Example from createlinks: my $event = "nethserver-samba-audit-update"; event_actions($event, qw( initialize-default-databases 00 nethserver-samba-audit-conf 02 ));

38

Chapter 3. Implementation

NethServer Documentation, Release 0

3.7 Migration from NethService/SME Server Migration is the process to convert a SME Server (or NethService) machine into a NethServer. 1. In the old host, create a full backup archive and move it to the new NethServer host. 2. In the new server, install all packages that cover the same features of the old one. 3. Explode the full backup archive on some directory (for instance /var/lib/migration) 4. Signal the event: signal-event migration-import /var/lib/migration

This step will require some time. 5. Search for any ERROR string in /var/log/messages

3.7.1 Coding conventions Most modules have already a migration action which handles the step automatically. A migration action: • must be named as -migrate • must be linked into migration-import event • must migrate old properties values to new ones • can copy original data files to the new location • must take care to apply the imported configuration, possibly using the -update event During migration some properties will not be imported: • UDPPort, TCPPort, UDPPorts, TCPPorts: all network ports will be reset to new defaults • DNS forwarder, green IP address, default gateway: these properties are filled up in bootstrap-console All e-smith databases are moved in /var/lib/nethserver/db directory. Code snippets A simple migrate action in perl.

#!/usr/bin/perl use esmith::DB::db; use esmith::event; use strict; my $event = shift; my $sourceDir = shift; my $esmithDbDir = 'home/e-smith/db'; my $errors = 0; if { die; } my $srcConfigDb = esmith::DB::db]>open\_ro(join('', $sourceDir, $esmithDbDir,'configuration')) || die my $dstConfigDb = esmith::DB::db->open || die; my $service = ‘ejabberd’; my $old = $srcConfigDb->get($service);

3.7. Migration from NethService/SME Server

39

NethServer Documentation, Release 0

my $new = $dstConfigDb->get || $dstConfigDb->new_record($service); $new->merge_props($old->props); # Apply configuration if( ! esmith::event::event_signal('nethserver-ejabberd-update')) { exit(1); } exit 0;

Remember to change the service name and add a license header. Add the migrate action to createlinks: #----------------------------------# actions for migration-import event #----------------------------------$event ="migration-import"; event_actions($event, '-migrate' => 60);

3.7.2 Packages Each packages with special migration notes is listed below. nethserver-base Properties not migrated: • PasswordSet • UnsavedChanges • bootstrap-console • dns • sysconfig • SystemMode • green network configuration • ActiveAccounts (moved to nethserver-directory, calculated on-the-fly) nethserver-backup No migration is possible. The backup must be reconfigured. nethserver-directory Home directories: user’s home directoriy migrates into /var/lib/nethserver/home, admin’s home directory migrates into /var/lib/nethserver/migration/admin, and a symlink is created in /root/admin-migration-. nethserver-hylafax After migration check the configuration of incoming fax notification.

40

Chapter 3. Implementation

NethServer Documentation, Release 0

nethserver-httpd The ibay-virtualhost relation has been designed differently from SME/NethService. An automatic migration is not always possible; the resulting configuration must be checked manually. The global-pw-remote’ case is currently not implemented in NethServer and is mapped as global-pw. The reason is we do not want make distinctions between internal/external connections. nethserver-mail-server During pseudonyms migration, • pseudonyms pointing to admin and shared accounts are mapped to postmaster, as any other account not existing in destination AccountsDB. Thus the resulting configuration requires post-migration supervision. • recursive pseudonyms (pointing to another pseudonym) are flattened and a relation with a user or group account record is established. Index of shared mailboxes is not migrated. Each user must re-share its own mail directory. To workaround this problem copy the original index file (/etc/dovecot/sharedmailbox/dict.db) to the new location (/var/lib/nethserver/vmail/shared-mailboxes.db) and restart dovecot. See http://wiki2.dovecot.org/SharedMailboxes/Shared for more information. Forbidden “\” in folder names

The dovecot plugin listplugin (http://wiki2.dovecot.org/Plugins/Listescape) is enabled, and uses backslash “\” as escape character. If original folder names contains “\”, run the following command after post-migration mail synchronization, to rename them:

find /var/lib/nethserver/vmail/ -type d -regex '.*\\.*' -prune | (while read -r SRC; do echo mv -iv "

nethserver-mail-filter No wildcards expansions are supported by nethserver-mail-filter UI interface; only full mail addresses or domain names. The migration action must map email addresses in the form *domain.tld to domain names, and log a warning whenever another form of wildcard expansion is used. Also recipient blacklists are not implemented and bayes DB is not migrated

3.8 Certificate Management nethserver-base provides a set of templates that output PEM-formatted certificate parts: • certificate/key RSA private key • certificate/crt public certificate • certificate/pem both key+crt parts Configuration is inside the configuration database. Example: pki=configuration KeyFile= CrtFile= ChainFile=

3.8. Certificate Management

41

NethServer Documentation, Release 0

CertificateDuration=365 CommonName=

A certificate consumer daemon should expand those templates to its own certificate paths, by installing the proper configuration under /etc/e-smith/templates.metadata. For instance nethserver-httpd adds the following template configuration: • /etc/e-smith/templates.metadata/etc/pki/tls/private/localhost.key TEMPLATE_PATH="certificate/key" OUTPUT_FILENAME="/etc/pki/tls/private/localhost.key" PERMS=0600 UID="root" GID="root"

• /etc/e-smith/templates.metadata/etc/pki/tls/certs/localhost.crt TEMPLATE_PATH="certificate/crt" OUTPUT_FILENAME="/etc/pki/tls/certs/localhost.crt" PERMS=0600 UID="root" GID="root"

Set OUTPUT_FILENAME, PERMS, UID and GID values according to daemon configuration.

3.8.1 Default behavior By default, CrtFile and KeyFile properties have empty values. In this case, nethserver-base generates a self-signed certificate during nethserver-base-update event. Default SELinux-aware certificate locations are: • /etc/pki/tls/private/NSRV.key: private key • /etc/pki/tls/certs/NSRV.crt: CA certificate A daily cron job checks certificate validity. certificate-update event is signaled.

If expired, the self-signed certificate is re-generated and

Default certificate duration is set to 365 days. To change it: db configuration setprop pki CertificateDuration 3650

The certificate Common Name is set to system FQDN. To override this value type: db configuration setprop pki CommonName custom.cn

3.9 Yum plugin The nethserver-yum package contains the nethserver_events which extends the post-RPM transaction hook. It executes the *-update event of each nethserver-* package involved in the transaction, preserving the order of RPM dependencies. Signalling update events after RPM cleaning up assures that any old event handler and template fragments have been removed. The configuration file is /etc/yum/pluginconf.d/nethserver.conf. Available options are:

42

Chapter 3. Implementation

NethServer Documentation, Release 0

• enabled: 1 (default) or 0, enable or disable the plugin • verbose: 1 or 0 (default), enable or disable debugging messages

3.9.1 Caching When NethServer is behind a proxy server you can force to bypass an intermediate caching by adding this option to the yum.conf file, under the main section: http_caching=none Be aware that yum.conf is a template and not using the cache can raise load on remote repository servers, use with care!

3.10 Backup NethServer has two kinds of backup: • configuration backup (nethserver-backup-config) • data backup (nethserver-backup-data) Configuration backup contains only system configuration files (passwd, config databases, etc). It’s scheduled to be executed every night and will create a new archive only if any file is changed in the last 24 hours. Backup libraries use conf.d directory behavior (see perldoc NethServer::Backup). When a backup is started, the system will search for all files in /etc/backup-config.d directory. This directory can contain .include and .exclude files. Each file contain a list of file to include/exclude into/from the backup. Example file /etc/backup-config.d/nethserver-base.include /etc/e-smith/templates-custom /etc/e-smith/templates-user-custom /etc/ssh /etc/sudoers /etc/passwd /etc/shadow /etc/group /etc/gshadow

Exclusions are evaluated after all inclusions. All libraries are inside the nethserver-backup-config package.

3.10.1 Log format Backup and restore actions will log all steps in a file using a parsable format. Each log line has the following format: - - -

Fields: • DATE_HOUR: date in ISO 8601 format (%Y-%m-%d) and time in 24 hour notation (%H:%M:%S) • TAG: can be START, STEP, SUCCESS or ERROR • MESSAGE: log message • EXIT_STATUS: (optional) the exit status of the process 3.10. Backup

43

NethServer Documentation, Release 0

3.10.2 Notifications Both nethserver-backup-config and nethserver-backup-data comes with two property to configure notification: • notify: enable or disable notification. Possible values: – always: send notification regardless of backup exit status – never: do not send any notification regardless of backup exit status – error: send notification only if an error occurs • notifyTo: notification mail destination address (default is: admin‘‘localhost)

3.10.3 Configuration backup The nethserver-backup-config package implements the backup of configuration and relies on the backup-config key inside the configuration database. Properties: * status : enable or disable the automatic backup, can be enabled or disabled. Default is enabled. * reinstall: enable or disable the reinstallation of RPMs during the restore process. Can be enabled or disabled. Default is enabled. Backup The main command is /sbin/e-smith/backup-config which starts the backup process (if enabled). The backup process has 3 steps: • pre-backup-config event: used to prepare data, for example a LDAP dump of users • backup-config-execute action: actually execute the backup if any file is changed in the last 24 hours. The backup file is saved in /var/lib/nethserver/backup/backup-config.tar.xz (see perldoc NethServer::BackupConfig) • post-backup-config event: used to post-process the backup file, for example to copy the backup to a remote server or encrypting the archive The configuration backup runs every night and it creates a new backup only if: • destination file does not exist • or new files are added or removed to/from the backup set • or content of any file inside the set is changed This package does not provide any default action in the pre-backup-config and post-backup-config events. But you can create a script inside the post-backup-config event to copy the configuration backup to a remote machine using, for example, the SSH protocol. Logs: • /var/log/backup-config.log: parsable log Restore The main command is /sbin/e-smith/restore-config which starts the restore process: • pre-restore-config event: used to prepare the system, for example stop a running service • restore-config-execute action: search for a backup file in the well-known directory (see above) and restore it

44

Chapter 3. Implementation

NethServer Documentation, Release 0

• post-restore-config event: used to apply restored configuration, for example reinstall packages and load the LDAP dump This package does not provide any action in the pre-restore-config event. Logs: • /var/log/restore-config.log: parsable log Customization Add custom include/exclude inside following files: • /etc/backup-config.d/custom.include • /etc/backup-config.d/custom.exclude

3.10.4 Data backup The‘nethserver-backup-data package‘‘ requires nethserver-backup-config. This module implements a traditional incremental/full backup. It uses the key backup-data inside configuration database. Properties: • status : enable or disable the automatic backup, can be enabled or disabled. Default is enabled. Regardless of this property, the backup is always executed if started manually • BackupTime: Default is 1:00

time of the scheduled backup.

Must be in the form ‘‘hh:mm.

• VFSType : set the backup medium, can be usb, cifs or nfs. • SMBShare: contains the Samba share name • SMBHost : host name of the SMB server • SMBLogin : login user for the SMB server • SMBPassword : password for the SMB server • USBLabel : contains the filesystem label • NFSHost : host name of the NFS server • NFShare : contains the NFS share name • Program : program used to perfrom the backup. Backup and restore processes will look for an action called respectively backup-data- and restore-data-. Default is: duplicity • Type : can be full or incremental. If full, a full backup will be executed every time. If incremental, a full backup will be executed once a week at FullDay, all other backups will be incremental • FullDay : number of day of the week when a full backup will be executed. Can be a number from 0 (Sunday) to 6 (Saturday). Defaults is 0. • Mount : directory where the share (or usb drive) will be mounted. Defaults is /mnt/backup • LogFile : output of the backup process. Default is /var/log/last-backup.log • VolSize : size of chunks in MB, if supported by Program. Default is 250 • CleanupOlderThan : time to retain backups, accept duplicity syntax (eg. 7D, 1M). Default is: never (keep all backups) 3.10. Backup

45

NethServer Documentation, Release 0

Supported VFSType: • cifs : save the backup on a remote SMB server. Authentication is mandatory. • nfs : save the backup on a remote NFS server. No authentication supported. • usb : save the backup on a USB device. The device must have a writable filesystem with a custom label. When the backup is started, the system will search for an USB device with the filesystem label saved in USBLabel. Backup The main command is /sbin/e-smith/backup-data which starts the backup process (if enabled). The backup is composed of three parts: • pre-backup-data event: prepare the system and mount the destination share • /etc/e-smith/events/actions/backup-data- action: execute the backup. This actions must implement full/incremental logic. The backup is directly saved on the mounted share (or usb device). • post-backup-data: umount share and cleanup. Actions in this event can also implement retention policies (currently not implemented). Logs: • /var/log/backup-data.log: parsable log • /var/log/last-backup.log: backup program output Indexing In the pre-backup-data event the disk analyzer (Duc) make an indexing of filesystem, useful to create the graphical tree. The name of the actions is /etc/e-smith/events/actions/nethserver-restore-data-duc-index and it compose the JSON file to create the navigable graphic tree. Customization Add custom include/exclude inside following files: • /etc/backup-data.d/custom.include • /etc/backup-data.d/custom.exclude Retention policy

All backups can be deleted after a certain amount of time. Cleanup process takes place in post-backup-data event. See CleanupOlderThan property. A log of cleanup action is saved in /var/log/last-cleanup.log. Duplicity

The default program used for backup is duplicity using the globbing file list feature. Encryption is disabled and duplicity cache is stored in /var/lib/nethserver/backup/duplicity/ directory. We plan to support all duplicity features in the near future.

46

Chapter 3. Implementation

NethServer Documentation, Release 0

See http://duplicity.nongnu.org/ for more information. Listing backup sets To list current backup sets: 1. Mount the backup directory 2. Query duplicity status 3. Umount the backup directory Just execute:

/etc/e-smith/events/actions/mount-`config getprop backup-data VFSType` duplicity collection-status --no-encryption --archive-dir /var/lib/nethserver/backup/duplicity/ file: /etc/e-smith/events/actions/umount-`config getprop backup-data VFSType`

Restore command line The main command is /sbin/e-smith/restore-data which starts the restore process: • pre-restore-data event: used to prepare the system (Eg. mysql stop) • restore-data- action: search for a backup in the configuration database and restore it • post-restore-data event: used to inform programs about new available data (eg. mysql restart) Restore grahic interface After the selection of the paths to restore, the main command called is /usr/libexec/nethserver/nethserver-restore-data-help that reads the list of paths to restore and creates a executable command to restore the directories. If the second option of restore was selected (Restored file without overwrite the existing files), after the restore in a temp directory, the script moves the restored directories in the correct paths. Logs: • /var/log/restore-data.log: parsable log • /var/log/restore.log: process output

3.11 Firewall and Gateway NethServer can work into two basic modes: • server mode: the system will be a standard host inside the network offering services like e-mail or file server. • gateway mode: the system is the gateway and firewall of the local network The system has an abstraction layer for firewall base functions, like opening ports for running services. Actually two implementations are available: • server mode: standard lokkit (default on CentOS) • gateway mode: advanced Shorewall configuration The gateway functionality is built around three modules: • nethserver-base: high-level abstraction of the firewall

3.11. Firewall and Gateway

47

NethServer Documentation, Release 0

• nethserver-firewall-base: Shorewall-based implementation • nethserver-lsm: link status monitor for multi-wan configurations

3.11.1 Roles and zones Each network interface has a role which maps to a firewall zone. The firewall has the following built-in zones, ordered from the most to the least privileged: • green: local network, it’s considered almost trusted. Hosts in this network can access any other zone. Hosts connected via VPN can be considered in green zone. • blue: guest network. Hosts in this network can access orange and red zones, can’t access green zone • orange: DMZ network. Hosts in this network can access red zone, can’t access green and blue zones • red: external/internet networks. Hosts in this network can access only firewall zone • a: (not implemented yet) traffic from/to this zone must be explicitly allowed There is also a special firewall zone which represents the firewall itself. The firewall can access any other zone. Each network interface with a configured role is a firewall zone. Roles are mapped to Shorewall zone as: • green -> loc • red -> net • blue -> blue • orange -> orang (in Shorewall, a zone name can’t be longer than 5 chars) • firewall -> FW Custom zone names are directly mapped to Shorewall respecting the limit of 5 chars. Red interfaces can be configured with static IP address or using DHCP. All other interfaces can be configured only with static IP addresses.

3.11.2 General configuration Properties of firewall key inside configuration db: • event: event to call when firewall-adjust event is fired • tc: traffic shape mode (see below) • ExternalPing: if enabled, allow ping responses on external interface • WanMode: multi-wan mode. Default is balance, can be: – balance: traffic is balanced among red interfaces in weighted mode – backup: traffic is routed via wan interface with maximum weight, all other interfaces are used as fallback • nfq: if enabled, traffic from external networks will be passed to NFQ and scanned with Snort. See IPS (Intrusion Prevention). • Policy: can be permissive or strict. See above. • MACValidation: if enabled, the firewall will check the traffic against a list of known MAC addresses (see: IP/MAC binding) • MACValidationPolicy: can be accept or drop. Default is drop. See man shorewall.conf for all valid values 48

Chapter 3. Implementation

NethServer Documentation, Release 0

• InterfaceRoleList: list of network interface roles configurable from web interface. green,red,blue,orange

Default is:

• CheckIP: comma-separeted list of IP monitored by LSM, to check if a provider is up or down • MaxNumberPacketLoss: number of maximum consecutive packets lost, over this threshold the provider is considered down • MaxPercentPacketLoss: percentage of maximum packet lost, over this threshold the provider is considered down • PingInterval: seconds between ICMP packets sent by LSM, default is 5 • NotifyWan: can be enabled or disabled, if enabled a mail is sent every time a provider changes its own state • NotifyWanFrom: sender address for mails sent if NotifyWAN is set to enabled • NotifyWanTo: recipient address for mails sent if NotifyWAN is set to enabled Example firewall=configuration event=nethserver-firewall-base-save tc=Simple nfq=disabled WanMode=balance Policy=permissive

3.11.3 Events The main event for firewall configuration is firewall-adjust. The event contains a single action which fires the event named in the property event inside the firewall key into the configuration database. Other events: • lokkit-save: base firewall implementation using lokkit • nethserver-firewall-base-save: firewall implementation using Shorewall • wan-uplink-update: fired when the status of an external interface changes The wan-uplink-event event takes at least two parameters: • provider name: name of the provider involved • action: can be up or down, describing the new provider status Example: signal-event wan-uplink-update down myisp

3.11.4 Policy For every network packet traveling between firewall zones, the system will evaluate a list of rules to allow/block the specific traffic. Policies are default firewall rules which will be applied only if no other rule matches the ongoing traffic. Firewall implements two standard policies: • Permissive: will enable all traffic from green (loc) zone to red (net) zone.

3.11. Firewall and Gateway

49

NethServer Documentation, Release 0

• Strict: will block all traffic from green (loc) zone to red (net) zone. Permitted traffic should be explicitly allowed. The firewall configures 4 default zones with built-in policies (see above). In the schema below, traffic is permitted from left to right and blocked from right to left: GREEN -> BLUE -> ORANGE -> RED To override a policy, you should create a firewall rule between zones.

3.11.5 IP/MAC binding When MACValidation option is enabled, the firewall analyzes all the traffic based on a well-known list of IPs associated to MAC addresses. If the host generating the traffic is not inside the list, MACValidationPolicy will be applied. The list of IP/MAC association is created from DHCP reservations. Thus, enabling MACValidation and leaving MACValidationPolicy set to drop, will block all traffic from hosts without a DHCP reservation.

3.11.6 Rules Firewall rules can allow or deny traffic matching certain conditions. Rules are saved inside the fwrules database as records of type rule. Each rule record has the following fields: • key: a unique key identifier • Position: integer sorting key • Src, Dst: {literal*|*reference} where – literal is an IP or CIDR – reference has the form prefix;value, where prefix can be a DB type (host, host-group, zone) or the string role, value is a DB key or an interface role name (green, red...) • Action: can be ACCEPT, DROP or REJECT – ACCEPT allows the traffic – REJECT denies the traffic, an ICMP port unreachable packet is sent to the source address – DROP discards the traffic without informing the source address (the connection will timeout) • Service: (optional) can be a service object or a port number. If a port number is used, both TCP and UDP protocols are matched. • Log: can be none or info. If value is info, all matched packets will be logged in /var/log/firewall.log. Defaults to none • status: can be enabled or disabled. Default is enabled • Description: (optional) Example of a rule accepting traffic: 1=rule Src=host;myhost Dst=192.168.1.2 Service=service;ssh Action=accept Position=32

50

Chapter 3. Implementation

NethServer Documentation, Release 0

Accept all traffic from myhost to myserver for ssh service (port 22):

db fwrules set 1 rule Src "host;myhost" Dst "host;myserver" Service ssh Action ACCEPT Log none status

Drop all traffic from 192.168.1.0/24 to 192.168.4.1 on TCP and UDP port 25: db fwrules set 2 rule Src

192.168.1.0/24 Dst 192.168.4.1 Service 22 Action DROP Log none status enab

Template Fragment Rules in the firewall can be added manually /etc/e-smith/templates/etc/shorewall/rules

by

a

template

fragment

in

the

folder

For example drop a file 40YourSpecificRule ## 40nethvoice { my $iax = $nethvoice{'AllowExternalIAX'} || 'disabled'; my $webrtc = $nethvoice{'AllowExternalWebRTC'} || 'disabled'; if ($iax eq 'enabled') { $OUT .= "# Enable IAX from red interfaces\n"; $OUT .= "?COMMENT Enable IAX from red interfaces\n"; $OUT .= "ACCEPT\tnet\t\$FW\tudp\t4569,5036\n"; } if ($webrtc eq 'enabled') { $OUT .= "# Enable WebRTC from red interfaces\n"; $OUT .= "?COMMENT Enable WebRTC from red interfaces\n"; $OUT .= "ACCEPT\tnet\t\$FW\ttcp\t8089\n"; } $OUT .= "?COMMENT\n"; }

You can use all the settings below but you might be interested by the shorewall documentation also at http://shorewall.net/manpages/shorewall-rules.html) • \t -> write a tab space (can be also written : $OUT .= "ACCEPT net $FW tcp 8089\n";) • ACCEPT -> allows the traffic • REJECT -> denies the traffic, an ICMP port unreachable packet is sent to the source address • DROP -> discards the traffic without informing the source address (the connection will timeout) • REDIRECT -> redirect the traffic to another firewall zone The target may optionally be followed by ”:” and a syslog log level (e.g, REJECT:info or Web(ACCEPT):debug). • loc -> green (Local network) • net -> red (Internet network)

3.11. Firewall and Gateway

51

NethServer Documentation, Release 0

• blue -> blue (Guest network) • orang -> orange (DMZ network) • $FW -> firewall • tcp -> tcp port (comma separated list of ports) • udp -> udp port (comma separated list of ports) then you must expand your templates and restart your firewall by : signal-event firewall-adjust

3.11.7 Firewall objects Firewall module uses objects to simplify rules creation. The use of objects is not mandatory but it’s strongly encouraged. Supported objects are: • Host • Group of host • Service • CIDR • Ip range • Zone A host is an already defined entry inside the hosts db, or a new key of type host: name=host IpAddress=IP Description=

A host-group is a group of hosts inside the hosts db. Hosts in a host-group should always be reachable using the same interface. For example: a group of host inside the LAN or on the Internet. A host-group db entry can be something like: name=host-group Members=host1,host2

A CIDR is a group of hosts defined as a CIDR network. It’s saved inside the hosts db: mycidr=cidr Address=192.168.100.0/24 Description=

A IP range is a group of hosts defined as a range of IP. It’s saved inside the hosts db: myrange=iprange Description= End=192.168.100.20 Start=192.168.100.10

A zone represents a network zone which can be associated to an interface or a set of IP address. A zone entry in networks database can be something like: name=zone Network=CIDR Interface=eth0

52

Chapter 3. Implementation

NethServer Documentation, Release 0

A configured network interface is automatically a zone. A service can have a protocol and one or more ports. A service entry in fwservices database can be something like: name=fwservice Protocol=TCP/UDP/TCPUDP/ICMP Ports=port/port range

Rules based on mac address It’s possible to create rules based on MAC address only using a template-custom. For example to block internet access to a host on local network using its MAC address:

mkdir -p /etc/e-smith/templates-custom/etc/shorewall/rules echo "DROP loc:~xx-xx-xx-xx-xx-xx net" > /etc/e-smith/templates-custom/etc/shorewall/ru

Where xx-xx-xx-xx-xx-xx is the MAC address to block. See man shorewall-rules for more information.

3.11.8 Port forwarding All port-forwards are saved inside the portforward db. Each record has: • key: auto-increment id • type: pf • protocol: tcp/udp • src: can be a port number or a range in the form xxxx:yyyy • dst: can be a port number, if empty the value of src is used • dstHost: destination host, can be an IP address or a hos firewall object • allow: allowed ip address or network, see SOURCE at http://www.shorewall.net/4.2/manpages/shorewallrules.html • status: enabled/disabled • oriDst: original destination ip, for example alias for a wan interface. If empty, the port forward is valid for all red interface • description: optional description

3.11.9 NAT 1:1 All NAT one-to-one configurations are stored in networks db. Each value is a new attribute for an existing alias key and the name of attribute is FwObjectNat that contains the reference of an associated host: eth1:0=alias FwObjectNat=host;host_name ipaddr=11.11.11.11

3.11. Firewall and Gateway

53

NethServer Documentation, Release 0

netmask=255.255.255.0 role=alias

During template-expanding phase, the associated host is mapping with referenced IP and added in shorewall nat configuration. The file is /etc/shorewall/nat. More information are available here: http://shorewall.net/NAT.htm

3.11.10 Traffic shaping Traffic shaping is implemented using the http://www.shorewall.net/simple_traffic_shaping.html

Shorewall

simple

traffic

shaping.

See:

The feature is controlled by tc property in firewall key from configuration db. Possible values are: • Simple: default • No: disabled See TC_ENABLED at http://shorewall.net/manpages/shorewall.conf.html . All traffic shaping rules are saved inside tc db. A record could be of type: • device: describe an interface • port: describe a rule for a port • ip: describe a rule for an ip (or MAC address) • helper: describe an helper rule (eg. sip) Device record: • key: interface name • In: inbound bandwidth in kbps • Out: outbound bandwidth in kbps • Priority: traffic priority, default is 2 • Description: optional description Port record: • key: port number • Priority: traffic priority • Proto: protocol name • Description: optional description Ip record: • key: host object, in the form host; • Priority: traffic priority • Description: optional description Helper record: • key: helper name • Priority: traffic priority 54

Chapter 3. Implementation

NethServer Documentation, Release 0

• Description: optional description For more information about helpers, see: http://www.shorewall.net/Helpers.html

3.11.11 Multi WAN NethServer firewall can handle 15 red (WAN) interfaces. Implementation uses Shorewall with LSM (Link Status Monitor). The LSM daemon takes care of monitoring WAN connections (interface) using ICMP traffic and it informs Shorewall about interface up/down events. Each interface can be checked using multiple IPs (see checkip property below). At least one IP must be reachable to mark the WAN connection as usable. If no IP is specified (recommended option), the system will uses well-known default IPs (Google DNS and OpenDNS). For each configured provider, LSM will send ping to a configured IP (checkip). When a provider status changes, the system will signal a wan-uplink-update event. Inside the event, the action nethserver-shorewall-wan-update invokes: • shorewall enable when a red interface is usable • shorewall disable then a red interface is not usable When an interface is disabled, all associated routes will be deleted. When a new TCP connection is started, a route is selected and all successive packets will always be routed via same interface. If the used interfaces goes down, the connection is closed. Actually two behaviors are implemented: balanced and active-backup. Balanced All red interfaces are simultaneously used accordingly to the configured weight (see below). Example: Given a connection A with weight 2, and connection B with weight 1, the firewall will route a double number of connections via A over B. Active-backup Red interfaces are ordered using the configured weight: higher the weight, higher the route priority. The interface with maximum weight will be the active connection, all other interfaces will be used if the active one goes down. Example Given 3 wan connections: • A with weight 3 • B with weight 2 • C with weight 1 All traffic is routed via A. On failure of A, all traffic is routed via B. When B goes down, C is used. Whenever A comes backup, all traffic is again routed through it.

3.11. Firewall and Gateway

55

NethServer Documentation, Release 0

Providers Providers are an abstraction over red interfaces (see man shorewall-providers). All providers must have a weight which is used to select the route for packets. A provider record inside the networks database has following properties: • key: name of provider • interface: associated red interface, it’s mandatory • weight: weight of connection expressed with an integer number, it’s mandatory • Description: (optional) custom description Example: myisp=provider interface=eth1 weight=5 Description=my fast provider

Multi WAN example 1. Configure two interfaces as red, for example eth1 and eth2 db networks setprop eth1 role red db networks setprop eth2 role red signal-event interface-update

2. Create two providers: db networks set firstisp provider interface eth1 weight 2 db networks set secondisp provider interface eth2 weight 1

3. Re-configure the firewall: signal-event firewall-adjust

See /var/log/firewall.log to check for up/down events. Routes can be checked using: shorewall show routing

Force traffic to a specific provider A mangle rule can route all matched network traffic through a specific provider. Mangle rules are record of type rule inside the tc database. For example, this rules will route all traffic to port 22 via the provider named myadsl: 1=rule Src=192.168.1.0/24 Dst=0.0.0.0/0 Service=fwservice;ssh Action=provider;myadsl status=enabled Position=2 Description=

56

Chapter 3. Implementation

NethServer Documentation, Release 0

Properties: • key: numeric id • Src: can be a ‘any’, role (execpt red), zone (not interface), host object, ip address, ip range or CIDR • Dst: can be a zone (not interface), host object, ip address, ip range or CIDR • Action: provider object, in the form of “provider;” • Service: (optional) can be a service object • status: can be enabled or disabled. Default is enabled • Position: integer sorting key • Description: (optional) A rule is ignore during template expansion if: • the source is red role • the destination is a role which is not red • source, destination and service are all set to any • the provider doesn’t exists • destination is set to any

3.11.12 Static routes Static routes are saved inside the routes database with a record of type static. Example: 8.8.4.4=static Description=My route Mask=255.255.255.255 Router=89.97.220.225

Each record has the following properties: • key: network address • Mask: network mask • Router: gateway for the network • Description: a custom description (optional) There is also a special type of static route called provider-static. These routes have the same properties as described above and are used to correctly route traffic for link monitor. This type of rules should never be manually edited.

3.12 IPS (Intrusion Prevention) The IPS (Intrusion Prevention System) module configures Snort using the netfilter queue (NFQUEUE). NFQUEUE is an iptables and ip6tables target which delegate the decision on packets to a userspace software. All inter-zone traffic will be analyzed by Snort itself. Snort rules are managed by Pulledpork. Alert logs are parsed by SnortALog and are available inside the Dashboard. Snort will analyze:

3.12. IPS (Intrusion Prevention)

57

NethServer Documentation, Release 0

• all inter-zone traffic, except traffic from firewall to local network • all traffic from port forwards

3.12.1 Manually enable/disable snort Snort will analyze network traffic only if nfqueue property inside the firewall key is set to enabled. The web interface will take care of this by assuring both firewall{nfqueue} and snort{status} are set to enabled (or disabled). Enabling: config setprop firewall nfqueue enabled config setprop snortd status enabled signal-event firewall-adjust signal-event nethserver-snort-save

Disabling: config setprop firewall nfqueue disabled config setprop snortd status disabled signal-event firewall-adjust signal-event nethserver-snort-save

3.12.2 Policies The package implements 4 policies. A policy is a set of rules which will change Snort behavior. Available policies are: • Connectivity: You run a lot of real time applications (VOIP, financial transactions, etc), and don’t want to run any rules that could affect the current performance of your gateway. Rules in this category make snort happy, additionally this category focuses on the high profile most likely to affect the largest number of people type of vulnerabilities. • Balanced: You are normal, you run normal stuff and you want normal security protections. This is the best policy to start from if you are new, old, or just plain average. If you don’t have any special requirements for super high speeds or super secure networks start here. • Security: You don’t care about dropping your bosses email, everything in your environment is tightly regulated and you don’t tolerate people stepping outside of your security policy. This policy hates on IM, P2P, vulnerabilities, malware, web apps that cause productivity loss, remote access, and just about anything not related to getting work done. If you run your network with an iron fist start here. • Expert: no policy is applied. All rules must be manually selected by the administrator (see pulledpork documentation). Changing the current policy to security: config setprop pulledpork Policy security signal-event nethserver-pulledpork-save

3.12.3 Troubleshooting When troubleshooting network traffic, just remember that Snort will intercept all the traffic. To temporary disable Snort use:

58

Chapter 3. Implementation

NethServer Documentation, Release 0

config setprop firewall nfqueue disabled signal-event firewall-adjust

To re-enable Snort: config setprop firewall nfqueue enabled signal-event firewall-adjust

3.13 Samba Accounts, files and printers server for a MS-Windows network based on “Samba”:http://samba.org.

3.13.1 Ibay profiles Note: Ibay profiles are not related to “Roaming profiles”! Ibays serve different purposes and smb.conf provides a lot of parameters to configure a Samba share. It’s difficult to find a combination of parameters that can fit all the possible requirements. Thus an ibay configuration adheres to a profile. An ibay profile is a smb.conf sub-template that expands a cohesive set of share parameters. Each ibay has SmbProfileType prop that selects the template to apply to the ibay. The template path must be placed into /etc/e-smith/templates/etc/smb.conf/ and prefixed by ibay-. Eg: default profile is located at /etc/e-smith/templates/etc/smb.conf/ibay-default. The default profile is applied if the given custom profile is not found. Configuration database Properties are saved inside the configuration database under the smb key: • ServerRole {WS,PDC,ADS} WorkStation, Primary Domain Controller, Active Directory member (available from version 1.3) roles. • Workgroup The name of the Domain for PDC and ADS roles; if empty use the first domain name component from DomainName key (uppercase). When ServerRole=WS the default value from Samba is used (i.e. WORKGROUP). • AdsRealm The name of the Kerberos realm of the Active Directory domain. If empty, use DomainName key value (uppercase). • DeadTime (days) See deadtime parameter in smb.conf(5) manpage. • LogonDrive [A-Z]: if empty, falls back on Z:. See logon drive parameter in smb.conf(5) manpage.

• RoamingProfiles {yes,no} enables “Windows roaming profiles”:http://wiki.samba.org/index.php/Samba*%26*Windows • WinsServerStatus if enabled act as a WINS server. • WinsServerIP ipaddress if WinsServerStatus is disabled, nmbd will register with the given WINS server. See wins server, remote announce, remote browse sync parameters in smb.conf(5) manpage. • UseCups {enabled,disabled} Use cups as printing server.

3.13. Samba

59

NethServer Documentation, Release 0

• UseClientDriver {yes,no} See use client driver parameter in smb.conf(5) manpage. • NetbiosAliasList See netbios aliases parameter in smb.conf(5) manpage. Example: smb=service ... Workgroup= AdsRealm= ServerRole=WS DeadTime=10080 LogonDrive= RoamingProfiles=no WinsServerStatus=disabled WinsServerIP= UseCups=enabled UseClientDriver=yes NetbiosAliasList=

3.13.2 Accounts database Only records with type ibay. Properties: • SmbStatus if enabled, activates ibay sharing through SMB protocol • SmbProfileType select the profile template to apply to the share (optional). The template path must be placed into /etc/e-smith/templates/etc/smb.conf/ and prefixed by ibay-. Eg: default profile is located at /etc/e-smith/templates/etc/smb.conf/ibay-default. • SmbRecycleBinStatus: enable or disable the recycle bin; when a file is deleted it is moved inside the recycle bin. • SmbShareBrowseable: controls the visibility of the shared folder, default is enabled. Example: iba1=ibay AclRead=domadmins,admin AclWrite=domadmins,admin Description=test GroupAccess=rw HttpStatus=disabled OtherAccess=r OwningGroup=locals SmbGuestAccessType=none SmbRecycleBinStatus=disabled SmbShareBrowseable=enabled SmbStatus=enabled

3.13.3 Active Directory domain member The Active Directory member role integrates AD users and groups into the system, through the winbind nsswitch module and provides some configuration hooks that other packages (i.e. [[nethserver-mail-server]]) can use to authenticate themselves through the “Kerberos protocol”:http://en.wikipedia.org/wiki/Kerberos*(protocol).

60

Chapter 3. Implementation

NethServer Documentation, Release 0

Join the AD domain This is the environment where I tested the AD join • NethServer 6.x host name nsrv2.ads1.tld • Windows Sever 2008 R2 Standard host name w2k8-ad.ads1.tld domain NSRV1 realm ADS1.TLD IP address 192.168.88.1 • Windows XP Professional 5.1 (2600) SP2 Thunderbird 10 ESR client Example: Server-manager graphical interface procedure (provided that ‘‘nethserver-hosts‘‘ and ‘‘nethserverntp‘‘ packages are installed) • Point your browser to the server-manager URL, here https://nsrv2.ads1.tld:980 • Authenticate as admin • In DNS & DHCP > DNS > Configure set 192.168.88.1 as primary DNS server • Set pool.ntp.org as timesource on AD. I followed the instructions about “Windows Time Service”:http://support.ntp.org/bin/view/Support/WindowsTimeService, from support.ntp.org. It seems that after setting the external time source, AD works as NTP server as well. • Back on server-manager, in Date and Time set pool.ntp.org, or the AD itself, 192.168.88.1, as NTP server • In Windows Network select Active Directory member role and fill required fields Example: Command line procedure • Login as root into nsrv2. • AD must be the DNS server: it contains SRV records necessary to the LDAP/Kerberos infrastructure. Set the AD controller as DNS name server: rpm -q nethserver-hosts || yum install nethserver-hosts config setprop dns NameServers 192.168.88.1 signal-event nethserver-hosts-update

• AD Kerberos usually requires that the difference between machine clocks is less than 5 minutes; -unfortunately upstream ntpd still does not support MSNTP authentication, thus we must configure both AD and NethServer to synchronize with an external time source: pool.ntp.org. For NethServer type: rpm -q nethserver-ntp || yum install nethserver-ntp config setprop ntpd status enabled NTPServer pool.ntp.org signal-event nethserver-ntp-save

• To set pool.ntp.org as timesource on AD I followed the instructions about “Windows Time Service”:http://support.ntp.org/bin/view/Support/WindowsTimeService, from support.ntp.org. • Now you’re ready to join AD domain. If you’re on a tty, nethserver-samba-adsjoin event will ask for administrator‘s password interactively, otherwise it will read the given file contents (/tmp/dummy here). config setprop smb Workgroup NSRV1 AdsRealm ADS1.TLD ServerRole ADS signal-event nethserver-samba-save touch /tmp/dummy signal-event nethserver-samba-adsjoin -u administrator -F /tmp/dummy join Enter administrator's password: Enter administrator's password: rm /tmp/dummy

3.13. Samba

61

NethServer Documentation, Release 0

Test if the join is OK with the following command: net -k ads testjoin

Join is OK getent passwd

Role description

When playing the Active Directory member role, some major changes happen on the system: • Users and groups from AD become available, through the winbind nsswitch module: :: getent passwd getent group • The system Kerberos keytab /etc/krb5.keytab is initialized and credentials are exported to each registered service keytab, respecting path and permissions requirements • Every hour a cronjob tests Kerberos TGTs for registered services, and renews any ticket that will soon expire • Every month a cronjob “changes the machine password”:http://blogs.technet.com/b/askds/archive/2009/02/15/test2.aspx, and keeps Kerberos keytab files for registered services updated with the new machine credentials Service configuration hooks

A service (i.e. dovecot) record in configuration DB can be extended with the following special props, that are read during the join operation, machine password renewal, and crojob tasks: dovecot=service ... KrbStatus=enabled KrbCredentialsCachePath= KrbKeytabPath=/var/lib/dovecot/krb5.keytab KrbPrimaryList=smtp,imap,pop KrbKeytabOwner= KrbKeytabPerms=

• KrbStatus {enabled,disabled} This is the main switch. If set to enabled a ticket credential cache file is kept valid by the hourly cronjob • KrbCredentialsCachePath The path of the credentials /tmp/krb5cc*, if service is also a system user.

cache.

It

defaults

to

• KrbKeytabPath Keytab file path. If empty, /var/lib/misc/nsrv-.keytab is assumed • KrbPrimaryList Defines the keytab contents. In Kerberos jargon a “primary” is the first part of the “principal”:http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5user/What-is-a-Kerberos-Principal*003f.html string, before the slash (/) character. Any primary in this list is exported to the keytab. • KrbKeytabOwner The unix file owner. Default is the service name. This is applied to both the credentials cache file and the keytab file. • KrbKeytabPerms The unix bit permissions in octal form. Default is 0400. This is applied to both the credentials cache file and the keytab file. The implementation is provided by source:nethserver-samba|root/usr/libexec/nethserver/smbads

62

Chapter 3. Implementation

NethServer Documentation, Release 0

Troubleshooting

# CHECK clock difference between NethServer and AD DC less than 5 minutes # CHECK NethServer uses only DC as DNS Samba join status Commands: • net ads info • net ads testjoin Does NSS/Winbind list AD users? Use: getent passwd

Does Dovecot see AD users? Use: doveadm user \*

Can I obtain Kerberos TGT? • Using keytab: kinit -t /etc/krb5.keytab -k '$'

• If keytab does not work, providing the secret machine password, stored in secrets.tdb, without the trailing NULL character: tdbdump /var/lib/samba/private/secrets.tdb | grep -A 1 SECRETS/MACHINE*PASSWORD kinit '$'

3.13.4 Changing hostname A new machine SID is generated when the server is in WS role and hostname changes. The SID is stored in secrets.tdb and in a new LDAP entry. When PDC role is then selected, one of the following scenarios applies: • if the domain (workgroup) was not previously created , a new sambaDomain entry is generated with the same SID of the new hostname; • if the workgroup was previously created, the old domain entry (and SID) is retained. In this scenario new user accounts retain the old workgroup SID. In other words, a new SID seems to be generated from hostname only. This differs from the official Samba documentation. “Security identifiers (SIDs)” section in “Updating Samba-3”:http://www.samba.org/samba/docs/man/SambaGuide/upgrades.html states about SIDs: The SID is generated in a nondeterminative manner. This means that each time it is generated for a particular combination of machine name (hostname) and domain name (workgroup), it will be different.

3.13. Samba

63

NethServer Documentation, Release 0

64

Chapter 3. Implementation

CHAPTER 4

Web interface

4.1 Web interface The web interface is the main configuration method for NethServer. The main goal is to hide the system configuration complexity behind a simple and clear interface. It runs on a special httpd instance and is accessible at: https://your_server:980 https://your_server/server-manager (if nethserver-httpd module is installed).

or

4.1.1 Nethgui framework Nethgui framework is provided with a set of components and basic classes to quickly build a modern web user interface. Its main goals are: • provide a simplified API for coding, localizing and realizing the web user interface and behaviour for project:NethServer based products. • perform user authentication and authorization; • offer test facilities to help module testing and debugging. Key features: • Object-oriented and embeddable • Plugin support • REST-aimed architecture • Almost no external dependencies • User roles • AJAX driven HTML5 Widget library • Unobtrusive JavaScript code • Compatible with console-based and mobile browsers NethServer Web User Interface is built upon Nethgui framework.

4.1.2 Design goals TODO

65

NethServer Documentation, Release 0

4.2 Creating web UI module In this page you can find a working example of a simple UI for a new module, but, first a little of theory.

4.2.1 Web UI structure All code is organized in three main directories: • /usr/share/nethesis/Nethgui: framework libraries (can be used for other projects) • /usr/share/nethesis/NethServer: actual implementation of modules UI • /usr/share/nethesis/nethserver-manager: web root directory Nethgui is a MVC framework that aims to abstract the developer from the backend and frontend. This means that you don’t have to deal with reading and writing properties from DB, neither care about HTML, CSS and JavaScript on the client side. You just write the controller. Of course, if you want, you can play with all three layers but it shouldn’t be necessary in the most of cases. Each module is composed by 4 parts: • Controller: /usr/share/nethesis/NethServer/Module/.php • View: /usr/share/nethesis/NethServer/Template/.php • Translation: /usr/share/nethesis/NethServer/Language//NethServer\_Module*.php • Inline help : /usr/share/nethesis/NethServer/Help/NethServer_Module*\ .html If needed, a module can add extra resources: • Specific authorization (JSON format): /usr/share/nethesis/NethServer/Authorization/.json - See Nethgui:Authorization • Custom CSS: /usr/share/nethesis/NethServer/Css/.css - See Including CSS • Custom JavaScript:

/usr/share/nethesis/NethServer/Js/.js - See Including JS

• Unit tests: /usr/share/nethesis/NethServer/Test/Unit/NethServer/Module/.php - See Nethgui unit test • Utility libraries: /usr/share/nethesis/NethServer/Tool/.php

66

Chapter 4. Web interface

NethServer Documentation, Release 0

4.2.2 Writing the code We add a UI to the package nethserver-ejabberd module. The UI will expose 2 properties of the ejabberd db key. Properties: • status: start and stop the ejabberd daemon on boot, can be enabled or disabled • WelcomeText: welcome text, can be anything Controller First, we create the controller which has 3 main functions: • initializeAttributes: handle module position in menu • initialize: bind the properties to the database and set the validator • onParametersSaved: apply the configuration Here is the controller (/usr/share/nethesis/NethServer/Module/Ejabber.php):

declareParameter('status', Validate::SERVICESTATUS, array('configuration', 'ejabberd', 'status')); // Bind 'WelcomeText' view parameter to 'WelcomeText' prop in ejabberd key of configuration d $this->declareParameter('WelcomeText', Validate::ANYTHING, array('configuration', 'ejabberd', } // Execute actions when saving parameters protected function onParametersSaved($changes) { // Signal nethserver-ejabberd-save event after saving props to db $this->getPlatform()->signalEvent('nethserver-ejabberd-save@post-process'); } }

View Show all fields using built-in functions. If needed, you can add extra HTML markup but remember that the output must be functional on any device (desktop, mobile, text browser, etc). Template (/usr/share/nethesis/NethServer/Template/Ejabber.php): header()->setAttribute('template', $T('Ejabber_Title')); // add simple panel echo $view->panel() //add 'status' parameter checkbox with value when checked and unchecked ->insert($view->checkbox('status', 'enabled')->setAttribute('uncheckedValue', 'disabled')) //add 'WelcomeText' text input field ->insert($view->textInput('WelcomeText')) ; // show submit and help buttons echo $view->buttonList($view::BUTTON_SUBMIT | $view::BUTTON_HELP);

4.2. Creating web UI module

67

NethServer Documentation, Release 0

Translation Translation files, are simple PHP files containing an associative array. All module language files are placed in /usr/share/nethesis/NethServer/Language/. Given a module with name “Test”, the english language file will be /usr/share/nethesis/NethServer/Language/en/NethServer_Module_Test.php. Warning messages about missing translations can be found in /var/log/messages ter Nethgui debug is enabled. To enable the debug, use index_dev.php on urls, https:///index_dev.php/en/.

afeg:

English translation (/usr/share/nethesis/NethServer/Language/en/NethServer_Module_Ejabber.php):