pre0115

4 years ago · 415ba84223
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -0,0 +1,746 @@
 # 1.8.7
 ### Compatibility-breaking changes
 * **IMPORTANT:** It is no longer possible to protect a group pad with a
  password. All API calls to `setPassword` or `isPasswordProtected` will fail.
  Existing group pads that were previously password protected will no longer be
  password protected. If you need fine-grained access control, you can restrict
  API session creation in your frontend service, or you can use plugins.
 * All workarounds for Microsoft Internet Explorer have been removed. IE might
  still work, but it is untested.
 * Plugin hook functions are now subject to new sanity checks. Buggy hook
  functions will cause an error message to be logged
 * Authorization failures now return 403 by default instead of 401
 * The `authorize` hook is now only called after successful authentication. Use
  the new `preAuthorize` hook if you need to bypass authentication
 * The `authFailure` hook is deprecated; use the new `authnFailure` and
  `authzFailure` hooks instead
 * The `indexCustomInlineScripts` hook was removed
 * The `client` context property for the `handleMessage` and
  `handleMessageSecurity` hooks has been renamed to `socket` (the old name is
  still usable but deprecated)
 * The `aceAttribClasses` hook functions are now called synchronously
 * The format of `ENTER`, `CREATE`, and `LEAVE` log messages has changed
 * Strings passed to `$.gritter.add()` are now expected to be plain text, not
  HTML. Use jQuery or DOM objects if you need formatting

 ### Notable new features
 * Users can now import without creating and editing the pad first
 * Added a new `readOnly` user setting that makes it possible to create users in
  `settings.json` that can read pads but not create or modify them
 * Added a new `canCreate` user setting that makes it possible to create users in
  `settings.json` that can modify pads but not create them
 * The `authorize` hook now accepts `readOnly` to grant read-only access to a pad
 * The `authorize` hook now accepts `modify` to grant modify-only (creation
  prohibited) access to a pad
 * All authentication successes and failures are now logged
 * Added a new `cookie.sameSite` setting that makes it possible to enable
  authentication when Etherpad is embedded in an iframe from another site
 * New `exportHTMLAdditionalContent` hook to include additional HTML content
 * New `exportEtherpadAdditionalContent` hook to include additional database
  content in `.etherpad` exports
 * New `expressCloseServer` hook to close Express when required
 * The `padUpdate` hook context now includes `revs` and `changeset`
 * `checkPlugins.js` has various improvements to help plugin developers
 * The HTTP request object (and therefore the express-session state) is now
  accessible from within most `eejsBlock_*` hooks
 * Users without a `password` or `hash` property in `settings.json` are no longer
  ignored, so they can now be used by authentication plugins
 * New permission denied modal and block ``permissionDenied``
 * Plugins are now updated to the latest version instead of minor or patches

 ### Notable fixes
 * Fixed rate limit accounting when Etherpad is behind a reverse proxy
 * Fixed typos that prevented access to pads via an HTTP API session
 * Fixed authorization failures for pad URLs containing a percent-encoded
  character
 * Fixed exporting of read-only pads
 * Passwords are no longer written to connection state database entries or logged
  in debug logs
 * When using the keyboard to navigate through the toolbar buttons the button
  with the focus is now highlighted
 * Fixed support for Node.js 10 by passing the `--experimental-worker` flag
 * Fixed export of HTML attributes within a line
 * Fixed occasional "Cannot read property 'offsetTop' of undefined" error in
  timeslider when "follow pad contents" is checked
 * socket.io errors are now displayed instead of silently ignored
 * Pasting while the caret is in a link now works (except for middle-click paste
  on X11 systems)
 * Removal of Microsoft Internet Explorer specific code
 * Import better handles line breaks and white space
 * Fix issue with ``createDiffHTML`` incorrect call of ``getInternalRevisionAText``
 * Allow additional characters in URLs
 * MySQL engine fix and various other UeberDB updates (See UeberDB changelog).
 * Admin UI improvements on search results (to remove duplicate items)
 * Removal of unused cruft from ``clientVars`` (``ip`` and ``userAgent``)


 ### Minor changes
 * Temporary disconnections no longer force a full page refresh
 * Toolbar layout for narrow screens is improved
 * Fixed `SameSite` cookie attribute for the `language`, `token`, and `pref`
  cookies
 * Fixed superfluous database accesses when deleting a pad
 * Expanded test coverage.
 * `package-lock.json` is now lint checked on commit
 * Various lint fixes/modernization of code

 # 1.8.6
 * IMPORTANT: This fixes a severe problem with postgresql in 1.8.5
 * SECURITY: Fix authentication and authorization bypass vulnerabilities
 * API: Update version to 1.2.15
 * FEATURE: Add copyPadWithoutHistory API (#4295)
 * FEATURE: Package more asset files to save http requests (#4286)
 * MINOR: Improve UI when reconnecting
 * TESTS: Improve tests

 # 1.8.5
 * IMPORTANT DROP OF SUPPORT: Drop support for IE.  Browsers now need async/await.
 * IMPORTANT SECURITY: Rate limit Commits when env=production
 * SECURITY: Non completed uploads no longer crash Etherpad
 * SECURITY: Log authentication requests
 * FEATURE: Support ES6 (migrate from Uglify-JS to Terser)
 * FEATURE: Improve support for non-cookie enabled browsers
 * FEATURE: New hooks for ``index.html``
 * FEATURE: New script to delete sessions.
 * FEATURE: New setting to allow import withing an author session on a pad
 * FEATURE: Checks Etherpad version on startup and notifies if update is available.  Also available in ``/admin`` interface.
 * FEATURE: Timeslider updates pad location to most recent edit
 * MINOR: Outdent UL/LI items on removal of list item
 * MINOR: Various UL/LI import/export bugs
 * MINOR: PDF export fix
 * MINOR: Front end tests no longer run (and subsequently error) on pull requests
 * MINOR: Fix issue with </li> closing a list before it opens
 * MINOR: Fix bug where large pads would fire a console error in timeslider
 * MINOR: Fix ?showChat URL param issue
 * MINOR: Issue where timeslider URI fails to be correct if padID is numeric
 * MINOR: Include prompt for clear authorship when entire document is selected
 * MINOR: Include full document aText every 100 revisions to make pad restoration on database curruption achievable
 * MINOR: Several Colibris CSS fixes
 * MINOR: Use mime library for mime types instead of hard-coded.
 * MINOR: Don't show "new pad button" if instance is read only
 * MINOR: Use latest NodeJS when doing Windows build
 * MINOR: Change disconnect logic to reconnect instead of silently failing
 * MINOR: Update SocketIO, async, jQuery and Mocha which were stuck due to stale code.
 * MINOR: Rewrite the majority of the ``bin`` scripts to use more modern syntax
 * MINOR: Improved CSS anomation through prefers-reduced-motion
 * PERFORMANCE: Use workers (where possible) to minify CSS/JS on first page request.  This improves initial startup times.
 * PERFORMANCE: Cache EJS files improving page load speed when maxAge > 0.
 * PERFORMANCE: Fix performance for large pads
 * TESTS: Additional test coverage for OL/LI/Import/Export
 * TESTS: Include Simulated Load Testing in CI.
 * TESTS: Include content collector tests to test contentcollector.js logic external to pad dependents.
 * TESTS: Include fuzzing import test.
 * TESTS: Ensure CI is no longer using any cache
 * TESTS: Fix various tests...
 * TESTS: Various additional Travis testing including libreoffice import/export

 # 1.8.4
 * FIX: fix a performance regression on MySQL introduced in 1.8.3
 * FIX: when running behind a reverse proxy and exposed in an inner directory, fonts and toolbar icons should now be visible. This is a regression introduced in 1.8.3
 * FIX: cleanups in the UI after the CSS rehaul of 1.8.3
 * MINOR: protect against bugged/stale UI elements after updates. An explicit cache busting via random query string is performed at each start. This needs to be replaced with hashed names in static assets.
 * MINOR: improved some tests
 * MINOR: fixed long-standing bugs in the maintenance tools in /bin (migrateDirtyDBtoRealDB, rebuildPad, convert, importSqlFile)

 # 1.8.3
 * FEATURE: colibris is now the default skin for new installs
 * FEATURE: improved colibris visuals, and migrated to Flexbox layout
 * FEATURE: skin variants: colibris skin colors can be easily customized. Visit http://127.0.0.1:9001/p/test#skinvariantsbuilder
 * REQUIREMENTS: minimum required Node version is **10.13.0 LTS**.
 * MINOR: stability fixes for the async migration in 1.8.0 (fixed many UnhandledPromiseRejectionWarning and the few remaining crashes)
 * MINOR: improved stability of import/export functionality
 * MINOR: fixed many small UI quirks (timeslider, import/export, chat)
 * MINOR: Docker images are now built & run in production mode by default
 * MINOR: reduced the size of the Docker images
 * MINOR: better documented cookies and configuration parameters of the Docker image
 * MINOR: better database support (especially MySQL)
 * MINOR: additional test coverage
 * MINOR: restored compatibility with ep_hash_auth
 * MINOR: migrate from swagger-node-express to openapi-backend
 * MINOR: honor the Accept-Language HTTP headers sent by browsers, eventually serving language variants
 * PERFORMANCE: correctly send HTTP/304 for minified files
 * SECURITY: bumped many dependencies. At the time of the release, this version has 0 reported vulnerabilities by npm audit
 * SECURITY: never send referrer when opening a link
 * SECURITY: rate limit imports and exports
 * SECURITY: do not allow pad import if a user never contributed to that pad
 * SECURITY: expose configuration parameter for limiting max import size

 *BREAKING CHANGE*: undoing the "clear authorship colors" command is no longer supported (see https://github.com/ether/etherpad-lite/issues/2802)
 *BREAKING CHANGE*: the visuals and CSS structure of the page was updated. Plugins may need a CSS rehaul

 # 1.8
 * SECURITY: change referrer policy so that Etherpad addresses aren't leaked when links are clicked (discussion: https://github.com/ether/etherpad-lite/pull/3636)
 * SECURITY: set the "secure" flag for the session cookies when served over SSL. From now on it will not be possible to serve the same instance both in cleartext and over SSL

 # 1.8-beta.1
 * FEATURE: code was migrated to `async`/`await`, getting rid of a lot of callbacks (see https://github.com/ether/etherpad-lite/issues/3540)
 * FEATURE: support configuration via environment variables
 * FEATURE: include an official Dockerfile in the main repository
 * FEATURE: support including plugins in custom Docker builds
 * FEATURE: conditional creation of users: when its password is null, a user is not created. This helps, for example, in advanced configuration of Docker images.
 * REQUIREMENTS: minimum required Node version is **8.9.0 LTS**. Release 1.8.3 will require at least Node **10.13.0** LTS
 * MINOR: in the HTTP API, allow URL parameters and POST bodies to co-exist
 * MINOR: fix Unicode bug in HTML export
 * MINOR: bugfixes to colibris chat window
 * MINOR: code simplification (avoided double negations, introduced early exits, ...)
 * MINOR: reduced the size of the Windows package
 * MINOR: upgraded the nodejs runtime to 10.16.3 in the Windows package
 * SECURITY: avoided XSS in IE11
 * SECURITY: the version is exposed in http header only when configured
 * SECURITY: updated vendored jQuery version
 * SECURITY: bumped dependencies

 # 1.7.5
 * FEATURE: introduced support for multiple skins. See https://etherpad.org/doc/v1.7.5/#index_skins
 * FEATURE: added a new, optional skin. It can be activated choosing `skinName: "colibris"` in `settings.json`
 * FEATURE: allow file import using LibreOffice
 * SECURITY: updated many dependencies. No known high or moderate risk dependencies remain.
 * SECURITY: generate better random pad names
 * FIX: don't nuke all installed plugins if `npm install` fails
 * FIX: improved LibreOffice export
 * FIX: allow debug mode on node versions >= 6.3
 * MINOR: started making Etherpad less dependent on current working directory when running
 * MINOR: started simplifying the code structure, flattening complex conditions
 * MINOR: simplified a bit the startup scripts

 *UPGRADE NOTES*: if you have custom files in `src/static/custom`, save them
 somewhere else, revert the directory contents, update to Etherpad 1.7.5, and
 finally put them back in their new location, uder `src/static/skins/no-skin`.

 # 1.7.0
 * FIX: `getLineHTMLForExport()` no longer produces multiple copies of a line. **WARNING**: this could potentially break some plugins
 * FIX: authorship of bullet points no longer changes when a second author edits them
 * FIX: improved Firefox compatibility (non printable keys)
 * FIX: `getPadPlainText()` was not working
 * REQUIREMENTS: minimum required Node version is 6.9.0 LTS. The next release will require at least Node 8.9.0 LTS
 * SECURITY: updated MySQL, Elasticsearch and PostgreSQL drivers
 * SECURITY: started updating deprecated code and packages
 * DOCS: documented --credentials, --apikey, --sessionkey. Better detailed contributors guidelines. Added a section on securing the installation

 # 1.6.6
 * FIX: line numbers are aligned with text again (broken in 1.6.4)
 * FIX: text entered between connection loss and reconnection was not saved
 * FIX: diagnostic call failed when etherpad was exposed in a subdirectory

 # 1.6.5
 * SECURITY: Escape data when listing available plugins
 * FIX: Fix typo in apicalls.js which prevented importing isValidJSONPName
 * FIX: fixed plugin dependency issue
 * FIX: Update iframe_editor.css
 * FIX: unbreak Safari iOS line wrapping

 # 1.6.4
 * SECURITY: Access Control bypass on /admin - CVE-2018-9845
 * SECURITY: Remote Code Execution through pad export - CVE-2018-9327
 * SECURITY: Remote Code Execution through JSONP handling - CVE-2018-9326
 * SECURITY: Pad data leak - CVE-2018-9325
 * Fix: Admin redirect URL
 * Fix: Various script Fixes
 * Fix: Various CSS/Style/Layout fixes
 * NEW: Improved Pad contents readability
 * NEW: Hook: onAccessCheck
 * NEW: SESSIONKEY and APIKey customizable path
 * NEW: checkPads script
 * NEW: Support "cluster mode"

 # 1.6.3
 * SECURITY: Update ejs
 * SECURITY: xss vulnerability when reading window.location.href
 * SECURITY: sanitize jsonp
 * NEW: Catch SIGTERM for graceful shutdown
 * NEW: Show actual applied text formatting for caret position
 * NEW: Add settings to improve scrolling of viewport on line changes

 # 1.6.2
 * NEW: Added pad shortcut disabling feature
 * NEW: Create option to automatically reconnect after a few seconds
 * Update: socket.io to 1.7.3
 * Update: l10n lib
 * Update: request to 2.83.0
 * Update: Node for windows to 8.9.0
 * Fix: minification of code

 # 1.6.1
 * NEW: Hook aceRegisterNonScrollableEditEvents to register events that shouldn't scroll
 * NEW: Added 'item' parameter to registerAceCommand Hook
 * NEW: Added LibreJS support
 * Fix: Crash on malformed export url
 * Fix: Re-enable editor after user is reconnected to server
 * Fix: minification
 * Other: Added 'no-referrer' for all pads
 * Other: Improved cookie security
 * Other: Fixed compatibility with nodejs 7
 * Other: Updates
  - socket.io to 1.6.0
  - express to 4.13.4
  - express-session to 1.13.0
  - clean-css to 3.4.12
  - uglify-js to 2.6.2
  - log4js to 0.6.35
  - cheerio to 0.20.0
  - ejs to 2.4.1
  - graceful-fs to 4.1.3
  - semver to 5.1.0
  - unorm to 1.4.1
  - jsonminify to 0.4.1
  - measured to 1.1.0
  - mocha to 2.4.5
  - supertest to 1.2.0
  - npm to 4.0.2
  - Node.js for Windows to 6.9.2

 # 1.6.0
 * SECURITY: Fix a possible xss attack in iframe link
 * NEW: Add a aceSelectionChanged hook to allow plugins to react when the cursor location changes.
 * NEW: Accepting Arrays on 'exportHtmlAdditionalTags' to handle attributes stored as ['key', 'value']
 * NEW: Allow admin to run on a sub-directory
 * NEW: Support version 5 of node.js
 * NEW: Update windows build to node version 4.4.3
 * NEW: Create setting to control if a new line will be indented or not
 * NEW: Add an appendText API
 * NEW: Allow LibreOffice to be used when exporting a pad
 * NEW: Create hook exportHtmlAdditionalTagsWithData
 * NEW: Improve DB migration performance
 * NEW: allow settings to be applied from the filesystem
 * NEW: remove applySettings hook and allow credentials.json to be part of core
 * NEW: Use exec to switch to node process
 * NEW: Validate incoming color codes
 * Fix: Avoid space removal when pasting text from word processor.
 * Fix: Removing style that makes editor scroll to the top on iOS without any action from the user
 * Fix: Fix API call appendChatMessage to send new message to all connected clients
 * Fix: Timeslider "Return to pad" button
 * Fix: Generating pad HTML with tags like <span data-TAG="VALUE"> instead of <TAG:VALUE>
 * Fix: Get git commit hash even if the repo only points to a bare repo.
 * Fix: Fix decode error if pad name contains special characters and is sanitized
 * Fix: Fix handleClientMessage_USER_* payloads not containing user info
 * Fix: Set language cookie on initial load
 * Fix: Timeslider Not Translated
 * Other: set charset for mysql connection in settings.json
 * Other: Dropped support for io.js
 * Other: Add support to store credentials in credentials.json
 * Other: Support node version 4 or higher
 * Other: Update uberDB to version 0.3.0

 # 1.5.7
 * NEW: Add support for intermediate CA certificates for ssl
 * NEW: Provide a script to clean up before running etherpad
 * NEW: Use ctrl+shift+1 to do a ordered list
 * NEW: Show versions of plugins on startup
 * NEW: Add author on padCreate and padUpdate hook
 * Fix: switchToPad method
 * Fix: Dead keys
 * Fix: Preserve new lines in copy-pasted text
 * Fix: Compatibility mode on IE
 * Fix: Content Collector to get the class of the DOM-node
 * Fix: Timeslider export links
 * Fix: Double prompt on file upload
 * Fix: setText() replaces the entire pad text
 * Fix: Accessibility features on embedded pads
 * Fix: Tidy HTML before abiword conversion
 * Fix: Remove edit buttons in read-only view
 * Fix: Disable user input in read-only view
 * Fix: Pads end with a single newline, rather than two newlines
 * Fix: Toolbar and chat for mobile devices

 # 1.5.6
 * Fix: Error on windows installations

 # 1.5.5
 * SECURITY: Also don't allow read files on directory traversal on minify paths
 * NEW: padOptions can be set in settings.json now
 * Fix: Add check for special characters in createPad API function
 * Fix: Middle click on a link in firefox don't paste text anymore
 * Fix: Made setPadRaw async to import larger etherpad files
 * Fix: rtl
 * Fix: Problem in older IEs
 * Other: Update to express 4.x
 * Other: Dropped support for node 0.8
 * Other: Update ejs to version 2.x
 * Other: Moved sessionKey from settings.json to a new auto-generated SESSIONKEY.txt file

 # 1.5.4
 * SECURITY: Also don't allow read files on directory traversal on frontend tests path

 # 1.5.3
 * NEW: Accessibility support for Screen readers, includes new fonts and keyboard shortcuts
 * NEW: API endpoint for Append Chat Message and Chat Backend Tests
 * NEW: Error messages displayed on load are included in Default Pad Text (can be supressed)
 * NEW: Content Collector can handle key values
 * NEW: getAttributesOnPosition Method
 * FIX: Firefox keeps attributes (bold etc) on cut/copy -> paste
 * Fix: showControls=false now works
 * Fix: Cut and Paste works...
 * SECURITY: Don't allow read files on directory traversal

 # 1.5.2
 * NEW: Support for node version 0.12.x
 * NEW: API endpoint saveRevision, getSavedRevisionCount and listSavedRevisions
 * NEW: setting to allow load testing
 * Fix: Rare scroll issue
 * Fix: Handling of custom pad path
 * Fix: Better error handling of imports and exports of type "etherpad"
 * Fix: Walking caret in chrome
 * Fix: Better handling for changeset problems
 * SECURITY Fix: Information leak for etherpad exports (CVE-2015-2298)

 # 1.5.1
 * NEW: High resolution Icon
 * NEW: Use HTTPS for plugins.json download
 * NEW: Add 'last update' column
 * NEW: Show users and chat at the same time
 * NEW: Support io.js
 * Fix: removeAttributeOnLine now works properly
 * Fix: Plugin search and list
 * Fix: Issue where unauthed request could cause error
 * Fix: Privacy issue with .etherpad export
 * Fix: Freeze deps to improve bisectability
 * Fix: IE, everything. IE is so broken.
 * Fix: Timeslider proxy
 * Fix: All backend tests pass
 * Fix: Better support for Export into HTML
 * Fix: Timeslider stars
 * Fix: Translation update
 * Fix: Check filesystem if Abiword exists
 * Fix: Docs formatting
 * Fix: Move Save Revision notification to a gritter message
 * Fix: UeberDB MySQL Timeout issue
 * Fix: Indented +9 list items
 * Fix: Don't paste on middle click of link
 * SECURITY Fix: Issue where a malformed URL could cause EP to disclose installation location

 # 1.5.0
 * NEW: Lots of performance improvements for page load times
 * NEW: Hook for adding CSS to Exports
 * NEW: Allow shardable socket io
 * NEW: Allow UI to show when attr/prop is applied (CSS)
 * NEW: Various scripts
 * NEW: Export full fidelity pads (including authors etc.)
 * NEW: Various front end tests
 * NEW: Backend tests
 * NEW: switchPad hook to instantly switch between pads
 * NEW: Various translations
 * NEW: Icon sets instead of images to provide quality high DPI experience
 * Fix: HTML Import blocking / hanging server
 * Fix: Export Bullet / Numbered lists HTML
 * Fix: Swagger deprecated warning
 * Fix: Bad session from crashing server
 * Fix: Allow relative settings path
 * Fix: Stop attributes being improperly assigned between 2 lines
 * Fix: Copy / Move Pad API race condition
 * Fix: Save all user preferences
 * Fix: Upgrade majority of dependency inc upgrade to SocketIO1+
 * Fix: Provide UI button to restore maximized chat window
 * Fix: Timeslider UI Fix
 * Fix: Remove Dokuwiki
 * Fix: Remove long paths from windows build (stops error during extract)
 * Fix: Various globals remvoed
 * Fix: Move all scripts into bin/
 * Fix: Various CSS bugfixes for Mobile devices
 * Fix: Overflow Toolbar
 * Fix: Line Attribute management

 # 1.4.1
 * NEW: Translations
 * NEW: userLeave Hook
 * NEW: Script to reinsert all DB values of a Pad
 * NEW: Allow for absolute settings paths
 * NEW: API: Get Pad ID from read Only Pad ID
 * NEW: Huge improvement on MySQL database read/write (InnoDB to MyISAM)
 * NEW: Hook for Export File Name
 * NEW: Preprocessor Hook for DOMLine attributes (allows plugins to wrap entire line contents)
 * Fix: Exception on Plugin Search and fix for plugins not being fetched
 * Fix: Font on innerdoc body can be arial on paste
 * Fix: Fix Dropping of messages in handleMessage
 * Fix: Don't use Abiword for HTML exports
 * Fix: Color issues with user Icon
 * Fix: Timeslider Button
 * Fix: Session Deletion error
 * Fix: Allow browser tabs to be cycled when focus is in editor
 * Fix: Various Editor issues with Easysync potentially entering forever loop on bad changeset

 # 1.4
 * NEW: Disable toolbar items through settings.json
 * NEW: Internal stats/metrics engine
 * NEW: Copy/Move Pad API functions
 * NEW: getAttributeOnSelection method
 * NEW: CSS function when an attribute is active on caret location
 * NEW: Various new eejs blocks
 * NEW: Ace afterEditHook
 * NEW: Import hook to introduce alternative export methods
 * NEW: preProcessDomLine allows Domline attributes to be processed before native attributes
 * Fix: Allow for lighter author colors
 * Fix: Improved randomness of session tokens
 * Fix: Don't panic if an author2session/group2session no longer exists
 * Fix: Gracefully fallback to related languages if chosen language is unavailable
 * Fix: Various changeset/stability bugs
 * Fix: Re-enable import buttons after failed import
 * Fix: Allow browser tabs to be cycled when in editor
 * Fix: Better Protocol detection
 * Fix: padList API Fix
 * Fix: Caret walking issue
 * Fix: Better settings.json parsing
 * Fix: Improved import/export handling
 * Other: Various whitespace/code clean-up
 * Other: .deb packaging creator
 * Other: More API Documentation
 * Other: Lots more translations
 * Other: Support Node 0.11

 # 1.3
 * NEW: We now follow the semantic versioning scheme!
 * NEW: Option to disable IP logging
 * NEW: Localisation updates from https://translatewiki.net.
 * Fix: Fix readOnly group pads
 * Fix: don't fetch padList on every request

 # 1.2.12
 * NEW: Add explanations for more disconnect scenarios
 * NEW: export sessioninfos so plugins can access it
 * NEW: pass pad in postAceInit hook
 * NEW: Add trustProxy setting. ALlows to make ep use X-forwarded-for as remoteAddress
 * NEW: userLeave hook (UNDOCUMENTED)
 * NEW: Plural macro for translations
 * NEW: backlinks to main page in Admin pages
 * NEW: New translations from translatewiki.net
 * SECURITY FIX: Filter author data sent to clients
 * FIX: Never keep processing a changeset if it's corrupted
 * FIX: Some client-side performance fixes for webkit browsers
 * FIX: Only execute listAllPads query on demand (not on start-up)
 * FIX: HTML import (don't crash on malformed or blank HTML input; strip title out of html during import)
 * FIX: check if uploaded file only contains ascii chars when abiword disabled
 * FIX: Plugin search in /admin/plugins
 * FIX: Don't create new pad if a non-existant read-only pad is accessed
 * FIX: Drop messages from unknown connections (would lead to a crash after a restart)
 * FIX: API: fix createGroupFor endpoint, if mapped group is deleted
 * FIX: Import form for other locales
 * FIX: Don't stop processing changeset queue if there is an error
 * FIX: Caret movement. Chrome detects blank rows line heights as incorrect
 * FIX: allow colons in password
 * FIX: Polish logging of client-side errors on the server
 * FIX: Username url param
 * FIX: Make start script POSIX ompatible


 # 1.2.11
 * NEW: New Hook for outer_ace dynamic css manager and author style hook
 * NEW: Bump log4js for improved logging
 * Fix: Remove URL schemes which don't have RFC standard
 * Fix: Fix safeRun subsequent restarts issue
 * Fix: Allow safeRun to pass arguements to run.sh
 * Fix: Include script for more efficient import
 * Fix: Fix sysv comptibile script
 * Fix: Fix client side changeset spamming
 * Fix: Don't crash on no-auth
 * Fix: Fix some IE8 errors
 * Fix: Fix authorship sanitation

 # 1.2.10
 * NEW: Broadcast slider is exposed in timeslider so plugins can interact with it
 * Fix: IE issue where pads wouldn't load due to missing console from i18n
 * Fix: console issue in collab client would error on cross domain embeds in IE
 * Fix: Only Restart Etherpad once plugin is installed
 * Fix: Only redraw lines that exist after drag and drop
 * Fix: Pasting into ordered list
 * Fix: Import browser detection
 * Fix: 2 Part Locale Specs
 * Fix: Remove language string from chat element
 * Fix: Make Saved revision Star fade back out on non Top frames
 * Other: Remove some cruft legacy JS from old Etherpad
 * Other: Express 3.1.2 breaks sessions, set Express to 3.1.0

 # 1.2.91
 * NEW: Authors can now send custom object messages to other Authors making 3 way conversations possible.  This introduces WebRTC plugin support.
 * NEW: Hook for Chat Messages Allows for Desktop Notification support
 * NEW: FreeBSD installation docs
 * NEW: Ctrl S for save revision makes the Icon glow for a few sconds.
 * NEW: Various hooks and expose the document ACE object
 * NEW: Plugin page revamp makes finding and installing plugins more sane.
 * NEW: Icon to enable sticky chat from the Chat box
 * Fix: Cookies inside of plugins
 * Fix: Don't leak event emitters when accessing admin/plugins
 * Fix: Don't allow user to send messages after they have been "kicked" from a pad
 * Fix: Refactor Caret navigation with Arrow and Pageup/down keys stops cursor being lost
 * Fix: Long lines in Firefox now wrap properly
 * Fix: Session Disconnect limit is increased from 10 to 20 to support slower restarts
 * Fix: Support Node 0.10
 * Fix: Log HTTP on DEBUG log level
 * Fix: Server wont crash on import fails on 0 file import.
 * Fix: Import no longer fails consistantly
 * Fix: Language support for non existing languages
 * Fix: Mobile support for chat notifications are now usable
 * Fix: Re-Enable Editbar buttons on reconnect
 * Fix: Clearing authorship colors no longer disconnects all clients
 * Other: New debug information for sessions

 # 1.2.9
 * Fix: MAJOR Security issue, where a hacker could submit content as another user
 * Fix: security issue due to unescaped user input
 * Fix: Admin page at /admin redirects to /admin/ now to prevent breaking relative links
 * Fix: indentation in chrome on linux
 * Fix: PadUsers API endpoint
 * NEW: A script to import data to all dbms
 * NEW: Add authorId to chat and userlist as a data attribute
 * NEW: Refactor and fix our frontend tests
 * NEW: Localisation updates


 # 1.2.81
 * Fix: CtrlZ-Y for Undo Redo
 * Fix: RTL functionality on contents & fix RTL/LTR tests and RTL in Safari
 * Fix: Various other tests fixed in Android

 # 1.2.8
 ! IMPORTANT: New setting.json value is required to automatically reconnect clients on disconnect
 * NEW: Use Socket IO for rooms (allows for pads to be load balanced with sticky rooms)
 * NEW: Plugins can now provide their own frontend tests
 * NEW: Improved server-side logging
 * NEW: Admin dashboard mobile device support and new hooks for Admin dashboard
 * NEW: Get current API version from API
 * NEW: CLI script to delete pads
 * Fix: Automatic client reconnection on disonnect
 * Fix: Text Export indentation now supports multiple indentations
 * Fix: Bugfix getChatHistory API method
 * Fix: Stop Chrome losing caret after paste is texted
 * Fix: Make colons on end of line create 4 spaces on indent
 * Fix: Stop the client disconnecting if a rev is in the wrong order
 * Fix: Various server crash issues based on rev in wrong order
 * Fix: Various tests
 * Fix: Make indent when on middle of the line stop creating list
 * Fix: Stop long strings breaking the UX by moving focus away from beginning of line
 * Fix: Redis findKeys support
 * Fix: padUsersCount no longer hangs server
 * Fix: Issue with two part locale specs not working
 * Fix: Make plugin search case insensitive
 * Fix: Indentation and bullets on text export
 * Fix: Resolve various warnings on dependencies during install
 * Fix: Page up / Page down now works in all browsers
 * Fix: Stop Opera browser inserting two new lines on enter keypress
 * Fix: Stop timeslider from showing NaN on pads with only one revision
 * Other: Allow timeslider tests to run and provide & fix various other frontend-tests
 * Other: Begin dropping referene to Lite.  Etherpad Lite is now named "Etherpad"
 * Other: Update to latest jQuery
 * Other: Change loading message asking user to please wait on first build
 * Other: Allow etherpad to use global npm installation (Safe since node 6.3)
 * Other: Better documentation for log rotation and log message handling



 # 1.2.7
 * NEW: notifications are now modularized and can be stacked
 * NEW: Visit a specific revision in the timeslider by suffixing #%revNumber% IE http://localhost/p/test/timeslider#12
 * NEW: Link to plugin on Admin page allows admins to easily see plugin details in a new window by clicking on the plugin name
 * NEW: Automatically see plugins that require update and be able to one click update
 * NEW: API endpoints for Chat .. getChatHistory, getChatHead
 * NEW: API endpoint to see a pad diff in HTML format from revision x to revision y .. createPadDiffHTML
 * NEW: Real time plugin search & unified menu UI for admin pages
 * Fix: MAJOR issue where server could be crashed by malformed client message
 * Fix: AuthorID is now included in padUsers API response
 * Fix: make docs
 * Fix: Timeslider UI bug with slider not being in position
 * Fix: IE8 language issue where it wouldn't load pads due to IE8 suckling on the bussum of hatrid
 * Fix: Import timeout issue
 * Fix: Import now works if Params are set in pad URL
 * Fix: Convert script
 * Other: Various new language strings and update/bugfixes of others
 * Other: Clean up the getParams functionality
 * Other: Various new EEJS blocks: index, timeslider, html etc.

 # 1.2.6
 * Fix: Package file UeberDB reference
 * New #users EEJS block for plugins

 # 1.2.5
 * Create timeslider EEJS blocks for plugins
 * Allow for "more messages" to be loaded in chat
 * Introduce better logging
 * API endpoint for "listAllPads"
 * Fix: Stop highlight of timeslider when dragging mouse
 * Fix: Time Delta on Timeslider make date update properly
 * Fix: Prevent empty chat messages from being sent
 * Fix: checkPad script
 * Fix: IE onLoad listener for i18n

 # 1.2.4
 * Fix IE console issue created in 1.2.3
 * Allow CI Tests to pass by ignoring timeslider test
 * Fix broken placeholders in locales
 * Fix extractPadData script
 * Fix documentation for checkToken
 * Fix hitting enter on form in admin/plugins

 # 1.2.3
 * Fix #1307: Chrome needs console.log to be called on console obj
 * Fix #1309: We had broken support for node v0.6 in the last release

 # 1.2.2
 * More translations and better language support.  See https://translatewiki.net/wiki/Translating:Etherpad_lite for more details
 * Add a checkToken Method to the API
 * Bugfix for Internal Caching issue that was causing some 404s on images.
 * Bugfix for IE Import
 * Bugfix for Node 0.6 compatibility
 * Bugfix for multiple cookie support
 * Bugfix for API when requireAuth is enabled.
 * Plugin page now shows plugin version #
 * Show color of Author in Chat messages
 * Allow plugin search by description
 * Allow for different socket IO transports
 * Allow for custom favicon path
 * Control S now does Create new Revision functionality
 * Focus on password when required
 * Frontend Timeslider test
 * Allow for basic HTML etc. import without abiword
 * Native HTTPS support

 # 1.2.1
 * Allow ! in urls inside the editor (Not Pad urls)
 * Allow comments in language files
 * More languages (Finish, Spanish, Bengali, Dutch) Thanks to TranslateWiki.net team.  See https://translatewiki.net/w/i.php?title=Special:MessageGroupStats&group=out-etherpad-lite for more details
 * Bugfix for IE7/8 issue with a JS error #1186
 * Bugfix windows package extraction issue and make the .zip file smaller
 * Bugfix group pad API export
 * Kristen Stewart is a terrible actress and Twilight sucks.

 # v1.2
 * Internationalization / Language / Translation support (i18n) with support for German/French
 * A frontend/client side testing framework and backend build tests
 * Customizable robots.txt
 * Customizable app title (finally you can name your epl instance!)
 * eejs render arguments are now passed on to eejs hooks through the newly introduced `renderContext` argument.
 * Plugin-specific settings in settings.json (finally allowing for things like a google analytics plugin)
 * Serve admin dashboard at /admin (still very limited, though)
 * Modify your settings.json through the newly created UI at /admin/settings
 * Fix: Import `<ol>` as `<ol>` and not as `<ul>`!
 * Added solaris compatibility (bin/installDeps.sh was broken on solaris)
 * Fix a bug with IE9 and Password Protected Pads using HTTPS

 # v1.1.5
 * We updated to express v3 (please [make sure](https://github.com/visionmedia/express/wiki/Migrating-from-2.x-to-3.x) your plugin works under express v3)
 * `userColor` URL parameter which sets the initial author color
 * Hooks for "padCreate", "padRemove", "padUpdate" and "padLoad" events
 * Security patches concerning the handling of messages originating from clients
 * Our database abstraction layer now natively supports couchDB, levelDB, mongoDB, postgres, and redis!
 * We now provide a script helping you to migrate from dirtyDB to MySQL
 * Support running Etherpad Lite behind IIS, using [iisnode](https://github.com/tjanczuk/iisnode/wiki)
 * LibreJS Licensing information in headers of HTML templates
 * Default port number to PORT env var, if port isn't specified in settings
 * Fix for `convert.js`
 * Raise upper char limit in chat to 999 characters
 * Fixes for mobile layout
 * Fixes for usage behind reverse proxy
 * Improved documentation
 * Fixed some opera style bugs
 * Update npm and fix some bugs, this introduces

 # v1.1
 * Introduced Plugin framework
 * Many bugfixes
 * Faster page loading
 * Various UI polishes
 * Saved Revisions
 * Read only Real time view
 * More API functionality

 # v 1.0.1

 * Updated MySQL driver, this fixes some problems with mysql
 * Fixed export,import and timeslider link when embed parameters are used
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -0,0 +1,133 @@
 # Contributor Guidelines
 (Please talk to people on the mailing list before you change this page, see our section on [how to get in touch](https://github.com/ether/etherpad-lite#get-in-touch))

 ## Pull requests

 * the commit series in the PR should be _linear_ (it **should not contain merge commits**). This is necessary because we want to be able to [bisect](https://en.wikipedia.org/wiki/Bisection_(software_engineering)) bugs easily. Rewrite history/perform a rebase if necessary
 * PRs should be issued against the **develop** branch: we never pull directly into **master**
 * PRs **should not have conflicts** with develop. If there are, please resolve them rebasing and force-pushing
 * when preparing your PR, please make sure that you have included the relevant **changes to the documentation** (preferably with usage examples)
 * contain meaningful and detailed **commit messages** in the form:
  ```
  submodule: description

  longer description of the change you have made, eventually mentioning the
  number of the issue that is being fixed, in the form: Fixes #someIssueNumber
  ```
 * if the PR is a **bug fix**:
  * the first commit in the series must be a test that shows the failure
  * subsequent commits will fix the bug and make the test pass
  * the final commit message should include the text `Fixes: #xxx` to link it to its bug report
 * think about stability: code has to be backwards compatible as much as possible. Always **assume your code will be run with an older version of the DB/config file**
 * if you want to remove a feature, **deprecate it instead**:
  * write an issue with your deprecation plan
  * output a `WARN` in the log informing that the feature is going to be removed
  * remove the feature in the next version
 * if you want to add a new feature, put it under a **feature flag**:
  * once the new feature has reached a minimal level of stability, do a PR for it, so it can be integrated early
  * expose a mechanism for enabling/disabling the feature
  * the new feature should be **disabled** by default. With the feature disabled, the code path should be exactly the same as before your contribution. This is a __necessary condition__ for early integration
 * think of the PR not as something that __you wrote__, but as something that __someone else is going to read__. The commit series in the PR should tell a novice developer the story of your thoughts when developing it

 ## How to write a bug report

 * Please be polite, we all are humans and problems can occur.
 * Please add as much information as possible, for example
  * client os(s) and version(s)
    * browser(s) and version(s), is the problem reproducible on different clients
    * special environments like firewalls or antivirus
  * host os and version
    * npm and nodejs version
    * Logfiles if available
  * steps to reproduce
  * what you expected to happen
  * what actually happened
 * Please format logfiles and code examples with markdown see github Markdown help below the issue textarea for more information.

 If you send logfiles, please set the loglevel switch DEBUG in your settings.json file:

 ```
 /* The log level we are using, can be: DEBUG, INFO, WARN, ERROR */
  "loglevel": "DEBUG",
 ```

 The logfile location is defined in startup script or the log is directly shown in the commandline after you have started etherpad.

 ## General goals of Etherpad
 To make sure everybody is going in the same direction:
 * easy to install for admins and easy to use for people
 * easy to integrate into other apps, but also usable as standalone
 * lightweight and scalable
 * extensible, as much functionality should be extendable with plugins so changes don't have to be done in core.
 Also, keep it maintainable. We don't wanna end up as the monster Etherpad was!

 ## How to work with git?
 * Don't work in your master branch.
 * Make a new branch for every feature you're working on. (This ensures that you can work you can do lots of small, independent pull requests instead of one big one with complete different features)
 * Don't use the online edit function of github (this only creates ugly and not working commits!)
 * Try to make clean commits that are easy readable (including descriptive commit messages!)
 * Test before you push. Sounds easy, it isn't!
 * Don't check in stuff that gets generated during build or runtime
 * Make small pull requests that are easy to review but make sure they do add value by themselves / individually

 ## Coding style
 * Do write comments. (You don't have to comment every line, but if you come up with something that's a bit complex/weird, just leave a comment. Bear in mind that you will probably leave the project at some point and that other people will read your code. Undocumented huge amounts of code are worthless!)
 * Never ever use tabs
 * Indentation: JS/CSS: 2 spaces; HTML: 4 spaces
 * Don't overengineer. Don't try to solve any possible problem in one step, but try to solve problems as easy as possible and improve the solution over time!
 * Do generalize sooner or later! (if an old solution, quickly hacked together, poses more problems than it solves today, refactor it!)
 * Keep it compatible. Do not introduce changes to the public API, db schema or configurations too lightly. Don't make incompatible changes without good reasons!
 * If you do make changes, document them! (see below)
 * Use protocol independent urls "//"

 ## Branching model / git workflow
 see git flow http://nvie.com/posts/a-successful-git-branching-model/

 ### `master` branch
 * the stable
 * This is the branch everyone should use for production stuff

 ### `develop`branch
 * everything that is READY to go into master at some point in time
 * This stuff is tested and ready to go out

 ### release branches
 * stuff that should go into master very soon
 * only bugfixes go into these (see http://nvie.com/posts/a-successful-git-branching-model/ for why)
 * we should not be blocking new features to develop, just because we feel that we should be releasing it to master soon. This is the situation that release branches solve/handle.

 ### hotfix branches
 * fixes for bugs in master

 ### feature branches (in your own repos)
 * these are the branches where you develop your features in
 * If it's ready to go out, it will be merged into develop

 Over the time we pull features from feature branches into the develop branch. Every month we pull from develop into master. Bugs in master get fixed in hotfix branches. These branches will get merged into master AND develop. There should never be commits in master that aren't in develop

 ## Documentation
 The docs are in the `doc/` folder in the git repository, so people can easily find the suitable docs for the current git revision.

 Documentation should be kept up-to-date. This means, whenever you add a new API method, add a new hook or change the database model, pack the relevant changes to the docs in the same pull request.

 You can build the docs e.g. produce html, using `make docs`. At some point in the future we will provide an online documentation. The current documentation in the github wiki should always reflect the state of `master` (!), since there are no docs in master, yet.

 ## Testing
 Front-end tests are found in the `tests/frontend/` folder in the repository. Run them by pointing your browser to `<yourdomainhere>/tests/frontend`.

 Back-end tests can be run from the `src` directory, via `npm test`.

 ## Things you can help with
 Etherpad is much more than software.  So if you aren't a developer then worry not, there is still a LOT you can do!  A big part of what we do is community engagement.  You can help in the following ways
 * Triage bugs (applying labels) and confirming their existence
 * Testing fixes (simply applying them and seeing if it fixes your issue or not) - Some git experience required
 * Notifying large site admins of new releases
 * Writing Changelogs for releases
 * Creating Windows packages
 * Creating releases
 * Bumping dependencies periodically and checking they don't break anything
 * Write proposals for grants
 * Co-Author and Publish CVEs
 * Work with SFC to maintain legal side of project
 * Maintain TODO page - https://github.com/ether/etherpad-lite/wiki/TODO#IMPORTANT_TODOS

--- a/+ 54
+++ b/+ 54
@ -0,0 +1,54 @@
 # Etherpad Lite Dockerfile
 #
 # https://github.com/ether/etherpad-lite
 #
 # Author: muxator

 FROM node:10-buster-slim
 LABEL maintainer="Etherpad team, https://github.com/ether/etherpad-lite"

 # plugins to install while building the container. By default no plugins are
 # installed.
 # If given a value, it has to be a space-separated, quoted list of plugin names.
 #
 # EXAMPLE:
 #   ETHERPAD_PLUGINS="ep_codepad ep_author_neat"
 ARG ETHERPAD_PLUGINS=

 # By default, Etherpad container is built and run in "production" mode. This is
 # leaner (development dependencies are not installed) and runs faster (among
 # other things, assets are minified & compressed).
 ENV NODE_ENV=production

 # Follow the principle of least privilege: run as unprivileged user.
 #
 # Running as non-root enables running this image in platforms like OpenShift
 # that do not allow images running as root.
 RUN useradd --uid 5001 --create-home etherpad

 RUN mkdir /opt/etherpad-lite && chown etherpad:0 /opt/etherpad-lite

 USER etherpad

 WORKDIR /opt/etherpad-lite

 COPY --chown=etherpad:0 ./ ./

 # install node dependencies for Etherpad
 RUN bin/installDeps.sh && \
 	rm -rf ~/.npm/_cacache

 # Install the plugins, if ETHERPAD_PLUGINS is not empty.
 #
 # Bash trick: in the for loop ${ETHERPAD_PLUGINS} is NOT quoted, in order to be
 # able to split at spaces.
 RUN for PLUGIN_NAME in ${ETHERPAD_PLUGINS}; do npm install "${PLUGIN_NAME}" || exit 1; done

 # Copy the configuration file.
 COPY --chown=etherpad:0 ./settings.json.docker /opt/etherpad-lite/settings.json

 # Fix permissions for root group
 RUN chmod -R g=u .

 EXPOSE 9001
 CMD ["node", "--experimental-worker", "node_modules/ep_etherpad-lite/node/server.js"]
--- a/+ 201
+++ b/+ 201
@ -0,0 +1,201 @@
                                 Apache License
                           Version 2.0, January 2004
                        http://www.apache.org/licenses/

   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

   1. Definitions.

      "License" shall mean the terms and conditions for use, reproduction,
      and distribution as defined by Sections 1 through 9 of this document.

      "Licensor" shall mean the copyright owner or entity authorized by
      the copyright owner that is granting the License.

      "Legal Entity" shall mean the union of the acting entity and all
      other entities that control, are controlled by, or are under common
      control with that entity. For the purposes of this definition,
      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or
      otherwise, or (ii) ownership of fifty percent (50%) or more of the
      outstanding shares, or (iii) beneficial ownership of such entity.

      "You" (or "Your") shall mean an individual or Legal Entity
      exercising permissions granted by this License.

      "Source" form shall mean the preferred form for making modifications,
      including but not limited to software source code, documentation
      source, and configuration files.

      "Object" form shall mean any form resulting from mechanical
      transformation or translation of a Source form, including but
      not limited to compiled object code, generated documentation,
      and conversions to other media types.

      "Work" shall mean the work of authorship, whether in Source or
      Object form, made available under the License, as indicated by a
      copyright notice that is included in or attached to the work
      (an example is provided in the Appendix below).

      "Derivative Works" shall mean any work, whether in Source or Object
      form, that is based on (or derived from) the Work and for which the
      editorial revisions, annotations, elaborations, or other modifications
      represent, as a whole, an original work of authorship. For the purposes
      of this License, Derivative Works shall not include works that remain
      separable from, or merely link (or bind by name) to the interfaces of,
      the Work and Derivative Works thereof.

      "Contribution" shall mean any work of authorship, including
      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"
      means any form of electronic, verbal, or written communication sent
      to the Licensor or its representatives, including but not limited to
      communication on electronic mailing lists, source code control systems,
      and issue tracking systems that are managed by, or on behalf of, the
      Licensor for the purpose of discussing and improving the Work, but
      excluding communication that is conspicuously marked or otherwise
      designated in writing by the copyright owner as "Not a Contribution."

      "Contributor" shall mean Licensor and any individual or Legal Entity
      on behalf of whom a Contribution has been received by Licensor and
      subsequently incorporated within the Work.

   2. Grant of Copyright License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      copyright license to reproduce, prepare Derivative Works of,
      publicly display, publicly perform, sublicense, and distribute the
      Work and such Derivative Works in Source or Object form.

   3. Grant of Patent License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      (except as stated in this section) patent license to make, have made,
      use, offer to sell, sell, import, and otherwise transfer the Work,
      where such license applies only to those patent claims licensable
      by such Contributor that are necessarily infringed by their
      Contribution(s) alone or by combination of their Contribution(s)
      with the Work to which such Contribution(s) was submitted. If You
      institute patent litigation against any entity (including a
      cross-claim or counterclaim in a lawsuit) alleging that the Work
      or a Contribution incorporated within the Work constitutes direct
      or contributory patent infringement, then any patent licenses
      granted to You under this License for that Work shall terminate
      as of the date such litigation is filed.

   4. Redistribution. You may reproduce and distribute copies of the
      Work or Derivative Works thereof in any medium, with or without
      modifications, and in Source or Object form, provided that You
      meet the following conditions:

      (a) You must give any other recipients of the Work or
          Derivative Works a copy of this License; and

      (b) You must cause any modified files to carry prominent notices
          stating that You changed the files; and

      (c) You must retain, in the Source form of any Derivative Works
          that You distribute, all copyright, patent, trademark, and
          attribution notices from the Source form of the Work,
          excluding those notices that do not pertain to any part of
          the Derivative Works; and

      (d) If the Work includes a "NOTICE" text file as part of its
          distribution, then any Derivative Works that You distribute must
          include a readable copy of the attribution notices contained
          within such NOTICE file, excluding those notices that do not
          pertain to any part of the Derivative Works, in at least one
          of the following places: within a NOTICE text file distributed
          as part of the Derivative Works; within the Source form or
          documentation, if provided along with the Derivative Works; or,
          within a display generated by the Derivative Works, if and
          wherever such third-party notices normally appear. The contents
          of the NOTICE file are for informational purposes only and
          do not modify the License. You may add Your own attribution
          notices within Derivative Works that You distribute, alongside
          or as an addendum to the NOTICE text from the Work, provided
          that such additional attribution notices cannot be construed
          as modifying the License.

      You may add Your own copyright statement to Your modifications and
      may provide additional or different license terms and conditions
      for use, reproduction, or distribution of Your modifications, or
      for any such Derivative Works as a whole, provided Your use,
      reproduction, and distribution of the Work otherwise complies with
      the conditions stated in this License.

   5. Submission of Contributions. Unless You explicitly state otherwise,
      any Contribution intentionally submitted for inclusion in the Work
      by You to the Licensor shall be under the terms and conditions of
      this License, without any additional terms or conditions.
      Notwithstanding the above, nothing herein shall supersede or modify
      the terms of any separate license agreement you may have executed
      with Licensor regarding such Contributions.

   6. Trademarks. This License does not grant permission to use the trade
      names, trademarks, service marks, or product names of the Licensor,
      except as required for reasonable and customary use in describing the
      origin of the Work and reproducing the content of the NOTICE file.

   7. Disclaimer of Warranty. Unless required by applicable law or
      agreed to in writing, Licensor provides the Work (and each
      Contributor provides its Contributions) on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
      implied, including, without limitation, any warranties or conditions
      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
      PARTICULAR PURPOSE. You are solely responsible for determining the
      appropriateness of using or redistributing the Work and assume any
      risks associated with Your exercise of permissions under this License.

   8. Limitation of Liability. In no event and under no legal theory,
      whether in tort (including negligence), contract, or otherwise,
      unless required by applicable law (such as deliberate and grossly
      negligent acts) or agreed to in writing, shall any Contributor be
      liable to You for damages, including any direct, indirect, special,
      incidental, or consequential damages of any character arising as a
      result of this License or out of the use or inability to use the
      Work (including but not limited to damages for loss of goodwill,
      work stoppage, computer failure or malfunction, or any and all
      other commercial damages or losses), even if such Contributor
      has been advised of the possibility of such damages.

   9. Accepting Warranty or Additional Liability. While redistributing
      the Work or Derivative Works thereof, You may choose to offer,
      and charge a fee for, acceptance of support, warranty, indemnity,
      or other liability obligations and/or rights consistent with this
      License. However, in accepting such obligations, You may act only
      on Your own behalf and on Your sole responsibility, not on behalf
      of any other Contributor, and only if You agree to indemnify,
      defend, and hold each Contributor harmless for any liability
      incurred by, or claims asserted against, such Contributor by reason
      of your accepting any such warranty or additional liability.

   END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.

      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "[]"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a
      file or class name and description of purpose be included on the
      same "printed page" as the copyright notice for easier
      identification within third-party archives.

   Copyright 2013 THE ETHERPAD FOUNDATION

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
--- a/+ 32
+++ b/+ 32
@ -0,0 +1,32 @@
 doc_sources = $(wildcard doc/*/*.md) $(wildcard doc/*.md)
 outdoc_files = $(addprefix out/,$(doc_sources:.md=.html))

 docassets = $(addprefix out/,$(wildcard doc/assets/*))

 VERSION = $(shell node -e "console.log( require('./src/package.json').version )")
 UNAME := $(shell uname -s)

 ensure_marked_is_installed:
 	set -eu; \
 	hash npm; \
 	if [ $(shell npm list --prefix bin/doc >/dev/null 2>/dev/null; echo $$?) -ne "0" ]; then \
 		npm ci --prefix=bin/doc; \
 	fi

 docs: ensure_marked_is_installed $(outdoc_files) $(docassets)

 out/doc/assets/%: doc/assets/%
 	mkdir -p $(@D)
 	cp $< $@

 out/doc/%.html: doc/%.md
 	mkdir -p $(@D)
 	node bin/doc/generate.js --format=html --template=doc/template.html $< > $@
 ifeq ($(UNAME),Darwin)
 	sed -i '' 's/__VERSION__/${VERSION}/' $@
 else
 	sed -i 's/__VERSION__/${VERSION}/' $@
 endif

 clean:
 	rm -rf out/
--- a/README.md
+++ b/README.md
@ -1,2 +1,154 @@
 # DasePad
 # A real-time collaborative editor for the web
 <a href="https://hub.docker.com/r/etherpad/etherpad"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/etherpad/etherpad"></a>

 [![Travis (.com)](https://api.travis-ci.com/ether/etherpad-lite.svg?branch=develop)](https://travis-ci.com/github/ether/etherpad-lite)

 ![Demo Etherpad Animated Jif](doc/images/etherpad_demo.gif "Etherpad in action")

 # About
 Etherpad is a real-time collaborative editor [scalable to thousands of simultaneous real time users](http://scale.etherpad.org/). It provides [full data export](https://github.com/ether/etherpad-lite/wiki/Understanding-Etherpad's-Full-Data-Export-capabilities) capabilities, and runs on _your_ server, under _your_ control.

 **[Try it out](https://video.etherpad.com)**

 # Installation

 ## Requirements
 - `nodejs` >= **10.13.0**.

 ## GNU/Linux and other UNIX-like systems

 ### Quick install on Debian/Ubuntu
 ```
 curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
 sudo apt install -y nodejs
 git clone --branch master https://github.com/ether/etherpad-lite.git && cd etherpad-lite && bin/run.sh
 ```

 ### Manual install
 You'll need git and [node.js](https://nodejs.org) installed (minimum required Node version: **10.13.0**).

 **As any user (we recommend creating a separate user called etherpad):**

 1. Move to a folder where you want to install Etherpad. Clone the git repository: `git clone --branch master git://github.com/ether/etherpad-lite.git`
 2. Change into the new directory containing the cloned source code: `cd etherpad-lite`
 3. run `bin/run.sh` and open <http://127.0.0.1:9001> in your browser.

 To update to the latest released version, execute `git pull origin`. The next start with `bin/run.sh` will update the dependencies.

 [Next steps](#next-steps).

 ## Windows

 ### Prebuilt Windows package
 This package runs on any Windows machine, but for development purposes, please do a manual install.

 1. [Download the latest Windows package](https://etherpad.org/#download)
 2. Extract the folder

 Run `start.bat` and open <http://localhost:9001> in your browser. You like it? [Next steps](#next-steps).

 ### Manually install on Windows
 You'll need [node.js](https://nodejs.org) and (optionally, though recommended) git.

 1. Grab the source, either
  - download <https://github.com/ether/etherpad-lite/zipball/master>
  - or `git clone --branch master https://github.com/ether/etherpad-lite.git`
 2. With a "Run as administrator" command prompt execute `bin\installOnWindows.bat`

 Now, run `start.bat` and open <http://localhost:9001> in your browser.

 Update to the latest version with `git pull origin`, then run `bin\installOnWindows.bat`, again.

 If cloning to a subdirectory within another project, you may need to do the following:

 1. Start the server manually (e.g. `node/node_modules/ep_etherpad-lite/node/server.js`)
 2. Edit the db `filename` in `settings.json` to the relative directory with the file (e.g. `application/lib/etherpad-lite/var/dirty.db`)
 3. Add auto-generated files to the main project `.gitignore`

 ## Docker container

 Find [here](doc/docker.md) information on running Etherpad in a container.

 # Next Steps

 ## Tweak the settings
 You can modify the settings in `settings.json`.
 If you need to handle multiple settings files, you can pass the path to a settings file to `bin/run.sh` using the `-s|--settings` option: this allows you to run multiple Etherpad instances from the same installation.
 Similarly, `--credentials` can be used to give a settings override file, `--apikey` to give a different APIKEY.txt file and `--sessionkey` to give a non-default SESSIONKEY.txt.
 **Each configuration parameter can also be set via an environment variable**, using the syntax `"${ENV_VAR}"` or `"${ENV_VAR:default_value}"`. For details, refer to `settings.json.template`.
 Once you have access to your `/admin` section settings can be modified through the web browser.

 If you are planning to use Etherpad in a production environment, you should use a dedicated database such as `mysql`, since the `dirtyDB` database driver is only for testing and/or development purposes.

 ## Secure your installation
 If you have enabled authentication in `users` section in `settings.json`, it is a good security practice to **store hashes instead of plain text passwords** in that file. This is _especially_ advised if you are running a production installation.

 Please install [ep_hash_auth plugin](https://www.npmjs.com/package/ep_hash_auth) and configure it.
 If you prefer, `ep_hash_auth` also gives you the option of storing the users in a custom directory in the file system, without having to edit `settings.json` and restart Etherpad each time.

 ## Customize functionalities with plugins

 ![Basic install](doc/images/etherpad_basic.png "Basic Installation")

 ![Full Features](doc/images/etherpad_full_features.png "You can add a lot of plugins !")

 Etherpad is very customizable through plugins. Instructions for installing themes and plugins can be found in [the plugin wiki article](https://github.com/ether/etherpad-lite/wiki/Available-Plugins).

 ## Getting the full features
 Run the following command in your Etherpad folder to get all of the features visible in the demo gif:

 ```
 npm install ep_headings2 ep_markdown ep_comments_page ep_align ep_page_view ep_font_color ep_webrtc ep_embedded_hyperlinks2
 ```

 ## Customize the style with skin variants

 Open <http://127.0.0.1:9001/p/test#skinvariantsbuilder> in your browser and start playing !

 ![Skin Variant](doc/images/etherpad_skin_variants.gif "Skin variants")

 ## Helpful resources
 The [wiki](https://github.com/ether/etherpad-lite/wiki) is your one-stop resource for Tutorials and How-to's.

 Documentation can be found in `doc/`.

 # Development

 ## Things you should know
 You can debug Etherpad using `bin/debugRun.sh`.

 You can run Etherpad quickly launching `bin/fastRun.sh`. It's convenient for developers and advanced users. Be aware that it will skip the dependencies update, so remember to run `bin/installDeps.sh` after installing a new dependency or upgrading version.

 If you want to find out how Etherpad's `Easysync` works (the library that makes it really realtime), start with this [PDF](https://github.com/ether/etherpad-lite/raw/master/doc/easysync/easysync-full-description.pdf) (complex, but worth reading).

 ## Contributing
 Read our [**Developer Guidelines**](https://github.com/ether/etherpad-lite/blob/master/CONTRIBUTING.md)

 # Get in touch
 The official channel for contacting the development team is via the [Github issues](https://github.com/ether/etherpad-lite/issues).

 For **responsible disclosure of vulnerabilities**, please write a mail to the maintainers (a.mux@inwind.it and contact@etherpad.org).
 Join the official [Etherpad Discord Channel](https://discord.com/invite/daEjfhw)

 # HTTP API
 Etherpad is designed to be easily embeddable and provides a [HTTP API](https://github.com/ether/etherpad-lite/wiki/HTTP-API)
 that allows your web application to manage pads, users and groups. It is recommended to use the [available client implementations](https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries) in order to interact with this API.

 OpenAPI (previously swagger) definitions for the API are exposed under `/api/openapi.json`.

 # jQuery plugin
 There is a [jQuery plugin](https://github.com/ether/etherpad-lite-jquery-plugin) that helps you to embed Pads into your website.

 # Plugin Framework
 Etherpad offers a plugin framework, allowing you to easily add your own features. By default your Etherpad is extremely light-weight and it's up to you to customize your experience. Once you have Etherpad installed you should [visit the plugin page](https://static.etherpad.org/) and take control.

 # Translations / Localizations  (i18n / l10n)
 Etherpad comes with translations into all languages thanks to the team at [TranslateWiki](https://translatewiki.net/).

 If you require translations in [plugins](https://static.etherpad.org/) please send pull request to each plugin individually.

 # FAQ
 Visit the **[FAQ](https://github.com/ether/etherpad-lite/wiki/FAQ)**.

 # License
 [Apache License v2](http://www.apache.org/licenses/LICENSE-2.0.html)
--- a/SECURITY.md
+++ b/SECURITY.md
@ -0,0 +1,5 @@
 # Security Policy

 ## Reporting a Vulnerability

 Please email contact@etherpad.org to report security related issues.
--- a/SESSIONKEY.txt
+++ b/SESSIONKEY.txt
@ -0,0 +1 @@
 10da85e41c3f1fd67a86550ded17d374ebf75dc7ffc5e427ee4aa513c3805ee6
--- a/bin/buildDebian.sh
+++ b/bin/buildDebian.sh
@ -0,0 +1,44 @@
 #!/usr/bin/env bash

 # IMPORTANT
 # Protect against misspelling a var and rm -rf /
 set -u
 set -e

 SRC=/tmp/etherpad-deb-src
 DIST=/tmp/etherpad-deb-dist
 SYSROOT=${SRC}/sysroot
 DEBIAN=${SRC}/DEBIAN

 rm -rf ${DIST}
 mkdir -p ${DIST}/

 rm -rf ${SRC}
 rsync -a bin/deb-src/ ${SRC}/
 mkdir -p ${SYSROOT}/opt/

 rsync --exclude '.git' -a . ${SYSROOT}/opt/etherpad/ --delete
 mkdir -p ${SYSROOT}/usr/share/doc
 cp README.md ${SYSROOT}/usr/share/doc/etherpad
 find ${SRC}/ -type d -exec chmod 0755 {} \;
 find ${SRC}/ -type f -exec chmod go-w {} \;
 chown -R root:root ${SRC}/

 let SIZE=$(du -s ${SYSROOT} | sed s'/\s\+.*//')+8
 pushd ${SYSROOT}/
 tar czf ${DIST}/data.tar.gz [a-z]*
 popd
 sed s"/SIZE/${SIZE}/" -i ${DEBIAN}/control
 pushd ${DEBIAN}
 tar czf ${DIST}/control.tar.gz *
 popd

 pushd ${DIST}/
 echo 2.0 > ./debian-binary

 find ${DIST}/ -type d -exec chmod 0755 {} \;
 find ${DIST}/ -type f -exec chmod go-w {} \;
 chown -R root:root ${DIST}/
 ar r ${DIST}/etherpad-1.deb debian-binary control.tar.gz data.tar.gz
 popd
 rsync -a ${DIST}/etherpad-1.deb ./
--- a/bin/buildForWindows.sh
+++ b/bin/buildForWindows.sh
@ -0,0 +1,63 @@
 #!/bin/sh

 pecho() { printf %s\n "$*"; }
 log() { pecho "$@"; }
 error() { log "ERROR: $@" >&2; }
 fatal() { error "$@"; exit 1; }
 is_cmd() { command -v "$@" >/dev/null 2>&1; }

 # Move to the folder where ep-lite is installed
 cd "$(dirname "$0")"/..

 # Is wget installed?
 is_cmd wget || fatal "Please install wget"

 # Is zip installed?
 is_cmd zip || fatal "Please install zip"

 # Is zip installed?
 is_cmd unzip || fatal "Please install unzip"

 START_FOLDER=$(pwd);
 TMP_FOLDER=$(mktemp -d)

 log "create a clean environment in $TMP_FOLDER..."
 cp -ar . "$TMP_FOLDER"
 cd "$TMP_FOLDER"
 rm -rf node_modules
 rm -f etherpad-lite-win.zip

 # setting NODE_ENV=production ensures that dev dependencies are not installed,
 # making the windows package smaller
 export NODE_ENV=production

 log "do a normal unix install first..."
 bin/installDeps.sh || exit 1

 log "copy the windows settings template..."
 cp settings.json.template settings.json

 log "resolve symbolic links..."
 cp -rL node_modules node_modules_resolved
 rm -rf node_modules
 mv node_modules_resolved node_modules

 log "download windows node..."
 cd bin
 wget "https://nodejs.org/dist/latest-erbium/win-x86/node.exe" -O ../node.exe

 log "remove git history to reduce folder size"
 rm -rf .git/objects

 log "remove windows jsdom-nocontextify/test folder"
 rm -rf "$TMP_FOLDER"/src/node_modules/wd/node_modules/request/node_modules/form-data/node_modules/combined-stream/test
 rm -rf "$TMP_FOLDER"/src/node_modules/nodemailer/node_modules/mailcomposer/node_modules/mimelib/node_modules/encoding/node_modules/iconv-lite/encodings/tables

 log "create the zip..."
 cd "$TMP_FOLDER"
 zip -9 -r "$START_FOLDER"/etherpad-lite-win.zip ./*

 log "clean up..."
 rm -rf "$TMP_FOLDER"

 log "Finished. You can find the zip in the Etherpad root folder, it's called etherpad-lite-win.zip"
--- a/bin/checkAllPads.js
+++ b/bin/checkAllPads.js
@ -0,0 +1,91 @@
 /*
 * This is a debug tool. It checks all revisions for data corruption
 */

 if (process.argv.length != 2) {
  console.error('Use: node bin/checkAllPads.js');
  process.exit(1);
 }

 // load and initialize NPM
 const npm = require('../src/node_modules/npm');
 npm.load({}, async () => {
  try {
    // initialize the database
    const settings = require('../src/node/utils/Settings');
    const db = require('../src/node/db/DB');
    await db.init();

    // load modules
    const Changeset = require('../src/static/js/Changeset');
    const padManager = require('../src/node/db/PadManager');

    // get all pads
    const res = await padManager.listAllPads();

    for (const padId of res.padIDs) {
      const pad = await padManager.getPad(padId);

      // check if the pad has a pool
      if (pad.pool === undefined) {
        console.error(`[${pad.id}] Missing attribute pool`);
        continue;
      }

      // create an array with key kevisions
      // key revisions always save the full pad atext
      const head = pad.getHeadRevisionNumber();
      const keyRevisions = [];
      for (let rev = 0; rev < head; rev += 100) {
        keyRevisions.push(rev);
      }

      // run through all key revisions
      for (const keyRev of keyRevisions) {
        // create an array of revisions we need till the next keyRevision or the End
        const revisionsNeeded = [];
        for (let rev = keyRev; rev <= keyRev + 100 && rev <= head; rev++) {
          revisionsNeeded.push(rev);
        }

        // this array will hold all revision changesets
        const revisions = [];

        // run through all needed revisions and get them from the database
        for (const revNum of revisionsNeeded) {
          const revision = await db.get(`pad:${pad.id}:revs:${revNum}`);
          revisions[revNum] = revision;
        }

        // check if the revision exists
        if (revisions[keyRev] == null) {
          console.error(`[${pad.id}] Missing revision ${keyRev}`);
          continue;
        }

        // check if there is a atext in the keyRevisions
        if (revisions[keyRev].meta === undefined || revisions[keyRev].meta.atext === undefined) {
          console.error(`[${pad.id}] Missing atext in revision ${keyRev}`);
          continue;
        }

        const apool = pad.pool;
        let atext = revisions[keyRev].meta.atext;

        for (let rev = keyRev + 1; rev <= keyRev + 100 && rev <= head; rev++) {
          try {
            const cs = revisions[rev].changeset;
            atext = Changeset.applyToAText(cs, atext, apool);
          } catch (e) {
            console.error(`[${pad.id}] Bad changeset at revision ${i} - ${e.message}`);
          }
        }
      }
      console.log('finished');
      process.exit(0);
    }
  } catch (err) {
    console.trace(err);
    process.exit(1);
  }
 });
--- a/bin/checkPad.js
+++ b/bin/checkPad.js
@ -0,0 +1,92 @@
 /*
 * This is a debug tool. It checks all revisions for data corruption
 */

 if (process.argv.length != 3) {
  console.error('Use: node bin/checkPad.js $PADID');
  process.exit(1);
 }

 // get the padID
 const padId = process.argv[2];

 // load and initialize NPM;
 const npm = require('../src/node_modules/npm');
 npm.load({}, async () => {
  try {
    // initialize database
    const settings = require('../src/node/utils/Settings');
    const db = require('../src/node/db/DB');
    await db.init();

    // load modules
    const Changeset = require('ep_etherpad-lite/static/js/Changeset');
    const padManager = require('../src/node/db/PadManager');

    const exists = await padManager.doesPadExists(padId);
    if (!exists) {
      console.error('Pad does not exist');
      process.exit(1);
    }

    // get the pad
    const pad = await padManager.getPad(padId);

    // create an array with key revisions
    // key revisions always save the full pad atext
    const head = pad.getHeadRevisionNumber();
    const keyRevisions = [];
    for (let rev = 0; rev < head; rev += 100) {
      keyRevisions.push(rev);
    }

    // run through all key revisions
    for (const keyRev of keyRevisions) {
      // create an array of revisions we need till the next keyRevision or the End
      const revisionsNeeded = [];
      for (let rev = keyRev; rev <= keyRev + 100 && rev <= head; rev++) {
        revisionsNeeded.push(rev);
      }

      // this array will hold all revision changesets
      const revisions = [];

      // run through all needed revisions and get them from the database
      for (const revNum of revisionsNeeded) {
        const revision = await db.get(`pad:${padId}:revs:${revNum}`);
        revisions[revNum] = revision;
      }

      // check if the pad has a pool
      if (pad.pool === undefined) {
        console.error('Attribute pool is missing');
        process.exit(1);
      }

      // check if there is an atext in the keyRevisions
      if (revisions[keyRev] === undefined || revisions[keyRev].meta === undefined || revisions[keyRev].meta.atext === undefined) {
        console.error(`No atext in key revision ${keyRev}`);
        continue;
      }

      const apool = pad.pool;
      let atext = revisions[keyRev].meta.atext;

      for (let rev = keyRev + 1; rev <= keyRev + 100 && rev <= head; rev++) {
        try {
          // console.log("check revision " + rev);
          const cs = revisions[rev].changeset;
          atext = Changeset.applyToAText(cs, atext, apool);
        } catch (e) {
          console.error(`Bad changeset at revision ${rev} - ${e.message}`);
          continue;
        }
      }
      console.log('finished');
      process.exit(0);
    }
  } catch (e) {
    console.trace(e);
    process.exit(1);
  }
 });
--- a/bin/checkPadDeltas.js
+++ b/bin/checkPadDeltas.js
@ -0,0 +1,111 @@
 /*
 * This is a debug tool. It checks all revisions for data corruption
 */

 if (process.argv.length != 3) {
  console.error('Use: node bin/checkPadDeltas.js $PADID');
  process.exit(1);
 }

 // get the padID
 const padId = process.argv[2];

 // load and initialize NPM;
 const expect = require('expect.js');
 const diff = require('diff');
 var async = require('async');

 const npm = require('../src/node_modules/npm');
 var async = require('ep_etherpad-lite/node_modules/async');
 const Changeset = require('ep_etherpad-lite/static/js/Changeset');

 npm.load({}, async () => {
  try {
    // initialize database
    const settings = require('../src/node/utils/Settings');
    const db = require('../src/node/db/DB');
    await db.init();

    // load modules
    const Changeset = require('ep_etherpad-lite/static/js/Changeset');
    const padManager = require('../src/node/db/PadManager');

    const exists = await padManager.doesPadExists(padId);
    if (!exists) {
      console.error('Pad does not exist');
      process.exit(1);
    }

    // get the pad
    const pad = await padManager.getPad(padId);

    // create an array with key revisions
    // key revisions always save the full pad atext
    const head = pad.getHeadRevisionNumber();
    const keyRevisions = [];
    for (var i = 0; i < head; i += 100) {
      keyRevisions.push(i);
    }

    // create an array with all revisions
    const revisions = [];
    for (var i = 0; i <= head; i++) {
      revisions.push(i);
    }

    let atext = Changeset.makeAText('\n');

    // run trough all revisions
    async.forEachSeries(revisions, (revNum, callback) => {
      // console.log('Fetching', revNum)
      db.db.get(`pad:${padId}:revs:${revNum}`, (err, revision) => {
        if (err) return callback(err);

        // check if there is a atext in the keyRevisions
        if (~keyRevisions.indexOf(revNum) && (revision === undefined || revision.meta === undefined || revision.meta.atext === undefined)) {
          console.error(`No atext in key revision ${revNum}`);
          callback();
          return;
        }

        try {
          // console.log("check revision ", revNum);
          const cs = revision.changeset;
          atext = Changeset.applyToAText(cs, atext, pad.pool);
        } catch (e) {
          console.error(`Bad changeset at revision ${revNum} - ${e.message}`);
          callback();
          return;
        }

        if (~keyRevisions.indexOf(revNum)) {
          try {
            expect(revision.meta.atext.text).to.eql(atext.text);
            expect(revision.meta.atext.attribs).to.eql(atext.attribs);
          } catch (e) {
            console.error(`Atext in key revision ${revNum} doesn't match computed one.`);
            console.log(diff.diffChars(atext.text, revision.meta.atext.text).map((op) => { if (!op.added && !op.removed) op.value = op.value.length; return op; }));
            // console.error(e)
            // console.log('KeyRev. :', revision.meta.atext)
            // console.log('Computed:', atext)
            callback();
            return;
          }
        }

        setImmediate(callback);
      });
    }, (er) => {
      if (pad.atext.text == atext.text) { console.log('ok'); } else {
        console.error('Pad AText doesn\'t match computed one! (Computed ', atext.text.length, ', db', pad.atext.text.length, ')');
        console.log(diff.diffChars(atext.text, pad.atext.text).map((op) => { if (!op.added && !op.removed) op.value = op.value.length; return op; }));
      }
      callback(er);
    });

    process.exit(0);
  } catch (e) {
    console.trace(e);
    process.exit(1);
  }
 });
--- a/bin/cleanRun.sh
+++ b/bin/cleanRun.sh
@ -0,0 +1,44 @@
 #!/bin/sh

 # Move to the folder where ep-lite is installed
 cd "$(dirname "$0")"/..

 # Source constants and usefull functions
 . bin/functions.sh

 #Was this script started in the bin folder? if yes move out
 if [ -d "../bin" ]; then
  cd "../"
 fi

 ignoreRoot=0
 for ARG in "$@"
 do
  if [ "$ARG" = "--root" ]; then
    ignoreRoot=1
  fi
 done

 #Stop the script if it's started as root
 if [ "$(id -u)" -eq 0 ] && [ $ignoreRoot -eq 0 ]; then
   echo "You shouldn't start Etherpad as root!"
   echo "Please type 'Etherpad rocks my socks' or supply the '--root' argument if you still want to start it as root"
   read rocks
   if [ ! $rocks = "Etherpad rocks my socks" ]
   then
     echo "Your input was incorrect"
     exit 1
   fi
 fi

 #Clean the current environment
 rm -rf src/node_modules

 #Prepare the environment
 bin/installDeps.sh "$@" || exit 1

 #Move to the node folder and start
 echo "Started Etherpad..."

 SCRIPTPATH=$(pwd -P)
 node $(compute_node_args) "${SCRIPTPATH}/node_modules/ep_etherpad-lite/node/server.js" "$@"
--- a/bin/convert.js
+++ b/bin/convert.js
@ -0,0 +1,391 @@
 const startTime = Date.now();
 const fs = require('fs');
 const ueberDB = require('../src/node_modules/ueberdb2');
 const mysql = require('../src/node_modules/ueberdb2/node_modules/mysql');
 const async = require('../src/node_modules/async');
 const Changeset = require('ep_etherpad-lite/static/js/Changeset');
 const randomString = require('ep_etherpad-lite/static/js/pad_utils').randomString;
 const AttributePool = require('ep_etherpad-lite/static/js/AttributePool');

 const settingsFile = process.argv[2];
 const sqlOutputFile = process.argv[3];

 // stop if the settings file is not set
 if (!settingsFile || !sqlOutputFile) {
  console.error('Use: node convert.js  $S E T T I N G S F I L E$ SQLOUTPUT');
  process.exit(1);
 }

 log('read settings file...');
 // read the settings file and parse the json
 const settings = JSON.parse(fs.readFileSync(settingsFile, 'utf8'));
 log('done');

 log('open output file...');
 const sqlOutput = fs.openSync(sqlOutputFile, 'w');
 const sql = 'SET CHARACTER SET UTF8;\n' +
          'CREATE TABLE IF NOT EXISTS `store` ( \n' +
          '`key` VARCHAR( 100 ) NOT NULL , \n' +
          '`value` LONGTEXT NOT NULL , \n' +
          'PRIMARY KEY (  `key` ) \n' +
          ') ENGINE = INNODB;\n' +
          'START TRANSACTION;\n\n';
 fs.writeSync(sqlOutput, sql);
 log('done');

 const etherpadDB = mysql.createConnection({
  host: settings.etherpadDB.host,
  user: settings.etherpadDB.user,
  password: settings.etherpadDB.password,
  database: settings.etherpadDB.database,
  port: settings.etherpadDB.port,
 });

 // get the timestamp once
 const timestamp = Date.now();

 let padIDs;

 async.series([
  // get all padids out of the database...
  function (callback) {
    log('get all padIds out of the database...');

    etherpadDB.query('SELECT ID FROM PAD_META', [], (err, _padIDs) => {
      padIDs = _padIDs;
      callback(err);
    });
  },
  function (callback) {
    log('done');

    // create a queue with a concurrency 100
    const queue = async.queue((padId, callback) => {
      convertPad(padId, (err) => {
        incrementPadStats();
        callback(err);
      });
    }, 100);

    // set the step callback as the queue callback
    queue.drain = callback;

    // add the padids to the worker queue
    for (let i = 0, length = padIDs.length; i < length; i++) {
      queue.push(padIDs[i].ID);
    }
  },
 ], (err) => {
  if (err) throw err;

  // write the groups
  let sql = '';
  for (const proID in proID2groupID) {
    const groupID = proID2groupID[proID];
    const subdomain = proID2subdomain[proID];

    sql += `REPLACE INTO store VALUES (${etherpadDB.escape(`group:${groupID}`)}, ${etherpadDB.escape(JSON.stringify(groups[groupID]))});\n`;
    sql += `REPLACE INTO store VALUES (${etherpadDB.escape(`mapper2group:subdomain:${subdomain}`)}, ${etherpadDB.escape(groupID)});\n`;
  }

  // close transaction
  sql += 'COMMIT;';

  // end the sql file
  fs.writeSync(sqlOutput, sql, undefined, 'utf-8');
  fs.closeSync(sqlOutput);

  log('finished.');
  process.exit(0);
 });

 function log(str) {
  console.log(`${(Date.now() - startTime) / 1000}\t${str}`);
 }

 let padsDone = 0;

 function incrementPadStats() {
  padsDone++;

  if (padsDone % 100 == 0) {
    const averageTime = Math.round(padsDone / ((Date.now() - startTime) / 1000));
    log(`${padsDone}/${padIDs.length}\t${averageTime} pad/s`);
  }
 }

 var proID2groupID = {};
 var proID2subdomain = {};
 var groups = {};

 function convertPad(padId, callback) {
  const changesets = [];
  const changesetsMeta = [];
  const chatMessages = [];
  const authors = [];
  let apool;
  let subdomain;
  let padmeta;

  async.series([
    // get all needed db values
    function (callback) {
      async.parallel([
        // get the pad revisions
        function (callback) {
          const sql = 'SELECT * FROM `PAD_CHAT_TEXT` WHERE NUMID = (SELECT `NUMID` FROM `PAD_CHAT_META` WHERE ID=?)';

          etherpadDB.query(sql, [padId], (err, results) => {
            if (!err) {
              try {
                // parse the pages
                for (let i = 0, length = results.length; i < length; i++) {
                  parsePage(chatMessages, results[i].PAGESTART, results[i].OFFSETS, results[i].DATA, true);
                }
              } catch (e) { err = e; }
            }

            callback(err);
          });
        },
        // get the chat entries
        function (callback) {
          const sql = 'SELECT * FROM `PAD_REVS_TEXT` WHERE NUMID = (SELECT `NUMID` FROM `PAD_REVS_META` WHERE ID=?)';

          etherpadDB.query(sql, [padId], (err, results) => {
            if (!err) {
              try {
                // parse the pages
                for (let i = 0, length = results.length; i < length; i++) {
                  parsePage(changesets, results[i].PAGESTART, results[i].OFFSETS, results[i].DATA, false);
                }
              } catch (e) { err = e; }
            }

            callback(err);
          });
        },
        // get the pad revisions meta data
        function (callback) {
          const sql = 'SELECT * FROM `PAD_REVMETA_TEXT` WHERE NUMID = (SELECT `NUMID` FROM `PAD_REVMETA_META` WHERE ID=?)';

          etherpadDB.query(sql, [padId], (err, results) => {
            if (!err) {
              try {
                // parse the pages
                for (let i = 0, length = results.length; i < length; i++) {
                  parsePage(changesetsMeta, results[i].PAGESTART, results[i].OFFSETS, results[i].DATA, true);
                }
              } catch (e) { err = e; }
            }

            callback(err);
          });
        },
        // get the attribute pool of this pad
        function (callback) {
          const sql = 'SELECT `JSON` FROM `PAD_APOOL` WHERE `ID` = ?';

          etherpadDB.query(sql, [padId], (err, results) => {
            if (!err) {
              try {
                apool = JSON.parse(results[0].JSON).x;
              } catch (e) { err = e; }
            }

            callback(err);
          });
        },
        // get the authors informations
        function (callback) {
          const sql = 'SELECT * FROM `PAD_AUTHORS_TEXT` WHERE NUMID = (SELECT `NUMID` FROM `PAD_AUTHORS_META` WHERE ID=?)';

          etherpadDB.query(sql, [padId], (err, results) => {
            if (!err) {
              try {
                // parse the pages
                for (let i = 0, length = results.length; i < length; i++) {
                  parsePage(authors, results[i].PAGESTART, results[i].OFFSETS, results[i].DATA, true);
                }
              } catch (e) { err = e; }
            }

            callback(err);
          });
        },
        // get the pad information
        function (callback) {
          const sql = 'SELECT JSON FROM `PAD_META` WHERE ID=?';

          etherpadDB.query(sql, [padId], (err, results) => {
            if (!err) {
              try {
                padmeta = JSON.parse(results[0].JSON).x;
              } catch (e) { err = e; }
            }

            callback(err);
          });
        },
        // get the subdomain
        function (callback) {
          // skip if this is no proPad
          if (padId.indexOf('$') == -1) {
            callback();
            return;
          }

          // get the proID out of this padID
          const proID = padId.split('$')[0];

          const sql = 'SELECT subDomain FROM pro_domains WHERE ID = ?';

          etherpadDB.query(sql, [proID], (err, results) => {
            if (!err) {
              subdomain = results[0].subDomain;
            }

            callback(err);
          });
        },
      ], callback);
    },
    function (callback) {
      // saves all values that should be written to the database
      const values = {};

      // this is a pro pad, let's convert it to a group pad
      if (padId.indexOf('$') != -1) {
        const padIdParts = padId.split('$');
        const proID = padIdParts[0];
        const padName = padIdParts[1];

        let groupID;

        // this proID is not converted so far, do it
        if (proID2groupID[proID] == null) {
          groupID = `g.${randomString(16)}`;

          // create the mappers for this new group
          proID2groupID[proID] = groupID;
          proID2subdomain[proID] = subdomain;
          groups[groupID] = {pads: {}};
        }

        // use the generated groupID;
        groupID = proID2groupID[proID];

        // rename the pad
        padId = `${groupID}$${padName}`;

        // set the value for this pad in the group
        groups[groupID].pads[padId] = 1;
      }

      try {
        const newAuthorIDs = {};
        const oldName2newName = {};

        // replace the authors with generated authors
        // we need to do that cause where the original etherpad saves pad local authors, the new (lite) etherpad uses them global
        for (var i in apool.numToAttrib) {
          var key = apool.numToAttrib[i][0];
          const value = apool.numToAttrib[i][1];

          // skip non authors and anonymous authors
          if (key != 'author' || value == '') continue;

          // generate new author values
          const authorID = `a.${randomString(16)}`;
          const authorColorID = authors[i].colorId || Math.floor(Math.random() * (exports.getColorPalette().length));
          const authorName = authors[i].name || null;

          // overwrite the authorID of the attribute pool
          apool.numToAttrib[i][1] = authorID;

          // write the author to the database
          values[`globalAuthor:${authorID}`] = {colorId: authorColorID, name: authorName, timestamp};

          // save in mappers
          newAuthorIDs[i] = authorID;
          oldName2newName[value] = authorID;
        }

        // save all revisions
        for (var i = 0; i < changesets.length; i++) {
          values[`pad:${padId}:revs:${i}`] = {changeset: changesets[i],
            meta: {
              author: newAuthorIDs[changesetsMeta[i].a],
              timestamp: changesetsMeta[i].t,
              atext: changesetsMeta[i].atext || undefined,
            }};
        }

        // save all chat messages
        for (var i = 0; i < chatMessages.length; i++) {
          values[`pad:${padId}:chat:${i}`] = {text: chatMessages[i].lineText,
            userId: oldName2newName[chatMessages[i].userId],
            time: chatMessages[i].time};
        }

        // generate the latest atext
        const fullAPool = (new AttributePool()).fromJsonable(apool);
        const keyRev = Math.floor(padmeta.head / padmeta.keyRevInterval) * padmeta.keyRevInterval;
        let atext = changesetsMeta[keyRev].atext;
        let curRev = keyRev;
        while (curRev < padmeta.head) {
          curRev++;
          const changeset = changesets[curRev];
          atext = Changeset.applyToAText(changeset, atext, fullAPool);
        }

        values[`pad:${padId}`] = {atext,
          pool: apool,
          head: padmeta.head,
          chatHead: padmeta.numChatMessages};
      } catch (e) {
        console.error(`Error while converting pad ${padId}, pad skipped`);
        console.error(e.stack ? e.stack : JSON.stringify(e));
        callback();
        return;
      }

      let sql = '';
      for (var key in values) {
        sql += `REPLACE INTO store VALUES (${etherpadDB.escape(key)}, ${etherpadDB.escape(JSON.stringify(values[key]))});\n`;
      }

      fs.writeSync(sqlOutput, sql, undefined, 'utf-8');
      callback();
    },
  ], callback);
 }

 /**
 * This parses a Page like Etherpad uses them in the databases
 * The offsets describes the length of a unit in the page, the data are
 * all values behind each other
 */
 function parsePage(array, pageStart, offsets, data, json) {
  let start = 0;
  const lengths = offsets.split(',');

  for (let i = 0; i < lengths.length; i++) {
    let unitLength = lengths[i];

    // skip empty units
    if (unitLength == '') continue;

    // parse the number
    unitLength = Number(unitLength);

    // cut the unit out of data
    const unit = data.substr(start, unitLength);

    // put it into the array
    array[pageStart + i] = json ? JSON.parse(unit) : unit;

    // update start
    start += unitLength;
  }
 }
--- a/bin/convertSettings.json.template
+++ b/bin/convertSettings.json.template
@ -0,0 +1,10 @@
 {
  "etherpadDB":
  {
    "host": "localhost",
    "port": 3306,
    "database": "etherpad",
    "user": "etherpaduser",
    "password": "yourpassword"
  }
 }
--- a/bin/createRelease.sh
+++ b/bin/createRelease.sh
@ -0,0 +1,203 @@
 #!/bin/bash
 #
 # WARNING: since Etherpad 1.7.0 (2018-08-17), this script is DEPRECATED, and
 #          will be removed/modified in a future version.
 #          It's left here just for documentation.
 #          The branching policies for releases have been changed.
 #
 # This script is used to publish a new release/version of etherpad on github
 #
 # Work that is done by this script:
 # ETHER_REPO:
 # - Add text to CHANGELOG.md
 # - Replace version of etherpad in src/package.json
 # - Create a release branch and push it to github
 # - Merges this release branch into master branch
 # - Creating the windows build and the docs
 # ETHER_WEB_REPO:
 # - Creating a new branch with the docs and the windows build
 # - Replacing the version numbers in the index.html
 # - Push this branch and merge it to master
 # ETHER_REPO:
 # - Create a new release on github

 printf "WARNING: since Etherpad 1.7.0 this script is DEPRECATED, and will be removed/modified in a future version.\n\n"
 while true; do
    read -p "Do you want to continue? This is discouraged. [y/N]" yn
    case $yn in
        [Yy]* ) break;;
        [Nn]* ) exit;;
        * ) printf "Please answer yes or no.\n\n";;
    esac
 done

 ETHER_REPO="https://github.com/ether/etherpad-lite.git"
 ETHER_WEB_REPO="https://github.com/ether/ether.github.com.git"
 TMP_DIR="/tmp/"

 echo "WARNING: You can only run this script if your github api token is allowed to create and merge branches on $ETHER_REPO and $ETHER_WEB_REPO."
 echo "This script automatically changes the version number in package.json and adds a text to CHANGELOG.md."
 echo "When you use this script you should be in the branch that you want to release (develop probably) on latest version. Any changes that are currently not committed will be committed."
 echo "-----"

 # Get the latest version
 LATEST_GIT_TAG=$(git tag | tail -n 1)

 # Current environment
 echo "Current environment: "
 echo "- branch: $(git branch | grep '* ')"
 echo "- last commit date: $(git show --quiet --pretty=format:%ad)"
 echo "- current version: $LATEST_GIT_TAG"
 echo "- temp dir: $TMP_DIR"

 # Get new version number
 # format: x.x.x
 echo -n "Enter new version (x.x.x): "
 read VERSION

 # Get the message for the changelogs
 read -p "Enter new changelog entries (press enter): "
 tmp=$(mktemp)
 "${EDITOR:-vi}" $tmp
 changelogText=$(<$tmp)
 echo "$changelogText"
 rm $tmp

 if [ "$changelogText" != "" ]; then
  changelogText="# $VERSION\n$changelogText"
 fi

 # get the token for the github api
 echo -n "Enter your github api token: "
 read API_TOKEN

 function check_api_token {
  echo "Checking if github api token is valid..."
  CURL_RESPONSE=$(curl --silent -i https://api.github.com/user?access_token=$API_TOKEN | iconv -f utf8)
  HTTP_STATUS=$(echo $CURL_RESPONSE | head -1 | sed -r 's/.* ([0-9]{3}) .*/\1/')
  [[ $HTTP_STATUS != "200" ]] && echo "Aborting: Invalid github api token" && exit 1
 }

 function modify_files {
  # Add changelog text to first line of CHANGELOG.md

  msg=""
  # source: https://unix.stackexchange.com/questions/9784/how-can-i-read-line-by-line-from-a-variable-in-bash#9789
  while IFS= read -r line
  do
    # replace newlines with literal "\n" for using with sed
    msg+="$line\n"
  done < <(printf '%s\n' "${changelogText}")

  sed -i "1s/^/${msg}\n/" CHANGELOG.md
  [[ $? != 0 ]] && echo "Aborting: Error modifying CHANGELOG.md" && exit 1

  # Replace version number of etherpad in package.json
  sed -i -r "s/(\"version\"[ ]*: \").*(\")/\1$VERSION\2/" src/package.json
  [[ $? != 0 ]] && echo "Aborting: Error modifying package.json" && exit 1
 }

 function create_release_branch {
  echo "Creating new release branch..."
  git rev-parse --verify release/$VERSION 2>/dev/null
  if [ $? == 0 ]; then
    echo "Aborting: Release branch already present"
    exit 1
  fi
  git checkout -b release/$VERSION
  [[ $? != 0 ]] && echo "Aborting: Error creating release branch" && exit 1

  echo "Committing CHANGELOG.md and package.json"
  git add CHANGELOG.md
  git add src/package.json
  git commit -m "Release version $VERSION"

  echo "Pushing release branch to github..."
  git push -u $ETHER_REPO release/$VERSION
  [[ $? != 0 ]] && echo "Aborting: Error pushing release branch to github" && exit 1
 }

 function merge_release_branch {
  echo "Merging release to master branch on github..."
  API_JSON=$(printf '{"base": "master","head": "release/%s","commit_message": "Merge new release into master branch!"}' $VERSION)
  CURL_RESPONSE=$(curl --silent -i -N --data "$API_JSON" https://api.github.com/repos/ether/etherpad-lite/merges?access_token=$API_TOKEN  | iconv -f utf8)
  echo $CURL_RESPONSE
  HTTP_STATUS=$(echo $CURL_RESPONSE | head -1 | sed -r 's/.* ([0-9]{3}) .*/\1/')
  [[ $HTTP_STATUS != "200" ]] && echo "Aborting: Error merging release branch on github" && exit 1
 }

 function create_builds {
  echo "Cloning etherpad-lite repo and ether.github.com repo..."
  cd $TMP_DIR
  rm -rf etherpad-lite ether.github.com
  git clone $ETHER_REPO --branch master
  git clone $ETHER_WEB_REPO
  echo "Creating windows build..."
  cd etherpad-lite
  bin/buildForWindows.sh
  [[ $? != 0 ]] && echo "Aborting: Error creating build for windows" && exit 1
  echo "Creating docs..."
  make docs
  [[ $? != 0 ]] && echo "Aborting: Error generating docs" && exit 1
 }

 function push_builds {
  cd $TMP_DIR/etherpad-lite/
  echo "Copying windows build and docs to website repo..."
  GIT_SHA=$(git rev-parse HEAD | cut -c1-10)
  mv etherpad-lite-win.zip $TMP_DIR/ether.github.com/downloads/etherpad-lite-win-$VERSION-$GIT_SHA.zip

  mv out/doc $TMP_DIR/ether.github.com/doc/v$VERSION

  cd $TMP_DIR/ether.github.com/
  sed -i "s/etherpad-lite-win.*\.zip/etherpad-lite-win-$VERSION-$GIT_SHA.zip/" index.html
  sed -i "s/$LATEST_GIT_TAG/$VERSION/g" index.html
  git checkout -b release_$VERSION
  [[ $? != 0 ]] && echo "Aborting: Error creating new release branch" && exit 1
  git add doc/
  git add downloads/
  git commit -a -m "Release version $VERSION"
  git push -u $ETHER_WEB_REPO release_$VERSION
  [[ $? != 0 ]] && echo "Aborting: Error pushing release branch to github" && exit 1
 }

 function merge_web_branch {
        echo "Merging release to master branch on github..."
        API_JSON=$(printf '{"base": "master","head": "release_%s","commit_message": "Release version %s"}' $VERSION $VERSION)
         CURL_RESPONSE=$(curl --silent -i -N --data "$API_JSON" https://api.github.com/repos/ether/ether.github.com/merges?access_token=$API_TOKEN | iconv -f utf8)
  echo $CURL_RESPONSE
  HTTP_STATUS=$(echo $CURL_RESPONSE | head -1 | sed -r 's/.* ([0-9]{3}) .*/\1/')
  [[ $HTTP_STATUS != "200" ]] && echo "Aborting: Error merging release branch" && exit 1
 }

 function publish_release {
  echo -n "Do you want to publish a new release on github (y/n)? "
  read PUBLISH_RELEASE
  if [ $PUBLISH_RELEASE = "y" ]; then
    # create a new release on github
    API_JSON=$(printf '{"tag_name": "%s","target_commitish": "master","name": "Release %s","body": "%s","draft": false,"prerelease": false}' $VERSION $VERSION $changelogText)
    CURL_RESPONSE=$(curl --silent -i -N --data "$API_JSON" https://api.github.com/repos/ether/etherpad-lite/releases?access_token=$API_TOKEN | iconv -f utf8)
    HTTP_STATUS=$(echo $CURL_RESPONSE | head -1 | sed -r 's/.* ([0-9]{3}) .*/\1/')
    [[ $HTTP_STATUS != "201" ]] && echo "Aborting: Error publishing release on github" && exit 1
  else
    echo "No release published on github!"
  fi
 }

 function todo_notification {
  echo "Release procedure was successful, but you have to do some steps manually:"
  echo "- Update the wiki at https://github.com/ether/etherpad-lite/wiki"
  echo "- Create a pull request on github to merge the master branch back to develop"
  echo "- Announce the new release on the mailing list, blog.etherpad.org and Twitter"
 }

 # Call functions
 check_api_token
 modify_files
 create_release_branch
 merge_release_branch
 create_builds
 push_builds
 merge_web_branch
 publish_release
 todo_notification
--- a/bin/createUserSession.js
+++ b/bin/createUserSession.js
@ -0,0 +1,47 @@
 /*
 * A tool for generating a test user session which can be used for debugging configs
 * that require sessions.
 */
 const m = (f) => `${__dirname}/../${f}`;

 const fs = require('fs');
 const path = require('path');
 const querystring = require('querystring');
 const request = require(m('src/node_modules/request'));
 const settings = require(m('src/node/utils/Settings'));
 const supertest = require(m('src/node_modules/supertest'));

 (async () => {
  const api = supertest(`http://${settings.ip}:${settings.port}`);

  const filePath = path.join(__dirname, '../APIKEY.txt');
  const apikey = fs.readFileSync(filePath, {encoding: 'utf-8'});

  let res;

  res = await api.get('/api/');
  const apiVersion = res.body.currentVersion;
  if (!apiVersion) throw new Error('No version set in API');
  const uri = (cmd, args) => `/api/${apiVersion}/${cmd}?${querystring.stringify(args)}`;

  res = await api.post(uri('createGroup', {apikey}));
  if (res.body.code === 1) throw new Error(`Error creating group: ${res.body}`);
  const groupID = res.body.data.groupID;
  console.log('groupID', groupID);

  res = await api.post(uri('createGroupPad', {apikey, groupID}));
  if (res.body.code === 1) throw new Error(`Error creating group pad: ${res.body}`);
  console.log('Test Pad ID ====> ', res.body.data.padID);

  res = await api.post(uri('createAuthor', {apikey}));
  if (res.body.code === 1) throw new Error(`Error creating author: ${res.body}`);
  const authorID = res.body.data.authorID;
  console.log('authorID', authorID);

  const validUntil = Math.floor(new Date() / 1000) + 60000;
  console.log('validUntil', validUntil);
  res = await api.post(uri('createSession', {apikey, groupID, authorID, validUntil}));
  if (res.body.code === 1) throw new Error(`Error creating session: ${res.body}`);
  console.log('Session made: ====> create a cookie named sessionID and set the value to',
      res.body.data.sessionID);
 })();
--- a/bin/deb-src/DEBIAN/control
+++ b/bin/deb-src/DEBIAN/control
@ -0,0 +1,9 @@
 Package: etherpad
 Version: 1.3
 Section: base
 Priority: optional
 Architecture: i386
 Installed-Size: SIZE
 Depends:
 Maintainer: John McLear <john@mclear.co.uk>
 Description: Etherpad is a collaborative editor.  
--- a/bin/deb-src/DEBIAN/postinst
+++ b/bin/deb-src/DEBIAN/postinst
@ -0,0 +1,7 @@
 #!/bin/bash
 # Start the services!

 service etherpad start
 echo "Give Etherpad about 3 minutes to install dependencies then visit http://localhost:9001 in your web browser"
 echo "To stop etherpad type 'service etherpad stop', To restart type 'service etherpad restart'".
 rm -f /tmp/etherpad.log /tmp/etherpad.err
--- a/bin/deb-src/DEBIAN/preinst
+++ b/bin/deb-src/DEBIAN/preinst
@ -0,0 +1,26 @@
 #!/bin/bash

 # Installs node if it isn't already installed
 #
 # Don't steamroll over a previously installed node version
 # TODO provide a local version of node?

 VER="0.10.4"
 ARCH="x86"
 if [ `arch | grep 64` ]
 then
  ARCH="x64"
 fi

 # TODO test version
 if [ ! -f /usr/local/bin/node ]
 then
  pushd /tmp
  wget -c "http://nodejs.org/dist/v $V E R / n o d e - v$ {VER}-linux-${ARCH}.tar.gz"
  rm -rf /tmp/node-v $V E R - l i n u x -$ {ARCH}
  tar xf node-v $V E R - l i n u x -$ {ARCH}.tar.gz -C /tmp/
  cp -a /tmp/node-v $V E R - l i n u x -$ {ARCH}/* /usr/local/
 fi

 # Create Etherpad user
 adduser --system etherpad
--- a/bin/deb-src/DEBIAN/prerm
+++ b/bin/deb-src/DEBIAN/prerm
@ -0,0 +1,4 @@
 #!/bin/bash

 # Stop the appserver:
 service etherpad stop || true
--- a/bin/deb-src/sysroot/etc/init/etherpad.conf
+++ b/bin/deb-src/sysroot/etc/init/etherpad.conf
@ -0,0 +1,28 @@
 description "etherpad"

 start on started networking
 stop on runlevel [!2345]

 env EPHOME=/opt/etherpad
 env EPLOGS=/var/log/etherpad
 env EPUSER=etherpad

 respawn

 pre-start script
    cd $EPHOME
    mkdir $EPLOGS                        ||true
    chown  $E P U S E R$ EPLOGS                ||true
    chmod 0755 $EPLOGS                   ||true
    chown -R  $E P U S E R$ EPHOME/var         ||true
     $E P H O M E / b i n / i n s t a l l D e p s . s h >>$ EPLOGS/error.log || { stop; exit 1; }
 end script

 script
  cd $EPHOME/
  exec su -s /bin/sh -c 'exec " $0 " "$ @"' $EPUSER -- node node_modules/ep_etherpad-lite/node/server.js \
                        >> $EPLOGS/access.log \
                        2>> $EPLOGS/error.log
  echo "Etherpad is running on http://localhost:9001 - To change settings edit /opt/etherpad/settings.json"

 end script
--- a/bin/debugRun.sh
+++ b/bin/debugRun.sh
@ -0,0 +1,18 @@
 #!/bin/sh

 # Move to the folder where ep-lite is installed
 cd "$(dirname "$0")"/..

 # Source constants and usefull functions
 . bin/functions.sh

 # Prepare the environment
 bin/installDeps.sh || exit 1

 echo "If you are new to debugging Node.js with Chrome DevTools, take a look at this page:"
 echo "https://medium.com/@paul_irish/debugging-node-js-nightlies-with-chrome-devtools-7c4a1b95ae27"
 echo "Open 'chrome://inspect' on Chrome to start debugging."

 # Use 0.0.0.0 to allow external connections to the debugger
 # (ex: running Etherpad on a docker container). Use default port # (9229)
 node $(compute_node_args) --inspect=0.0.0.0:9229 node_modules/ep_etherpad-lite/node/server.js "$@"
--- a/bin/deleteAllGroupSessions.js
+++ b/bin/deleteAllGroupSessions.js
@ -0,0 +1,51 @@
 /*
 * A tool for deleting ALL GROUP sessions Etherpad user sessions from the CLI,
 * because sometimes a brick is required to fix a face.
 */

 const request = require('../src/node_modules/request');
 const settings = require(`${__dirname}/../tests/container/loadSettings`).loadSettings();
 const supertest = require(`${__dirname}/../src/node_modules/supertest`);
 const api = supertest(`http://${settings.ip}:${settings.port}`);
 const path = require('path');
 const fs = require('fs');

 // get the API Key
 const filePath = path.join(__dirname, '../APIKEY.txt');
 const apikey = fs.readFileSync(filePath, {encoding: 'utf-8'});

 // Set apiVersion to base value, we change this later.
 let apiVersion = 1;
 let guids;

 // Update the apiVersion
 api.get('/api/')
    .expect((res) => {
      apiVersion = res.body.currentVersion;
      if (!res.body.currentVersion) throw new Error('No version set in API');
      return;
    })
    .then(() => {
      const guri = `/api/${apiVersion}/listAllGroups?apikey=${apikey}`;
      api.get(guri)
          .then((res) => {
            guids = res.body.data.groupIDs;
            guids.forEach((groupID) => {
              const luri = `/api/${apiVersion}/listSessionsOfGroup?apikey=${apikey}&groupID=${groupID}`;
              api.get(luri)
                  .then((res) => {
                    if (res.body.data) {
                      Object.keys(res.body.data).forEach((sessionID) => {
                        if (sessionID) {
                          console.log('Deleting', sessionID);
                          const duri = `/api/${apiVersion}/deleteSession?apikey=${apikey}&sessionID=${sessionID}`;
                          api.post(duri); // deletes
                        }
                      });
                    } else {
                      // no session in this group.
                    }
                  });
            });
          });
    });
--- a/bin/deletePad.js
+++ b/bin/deletePad.js
@ -0,0 +1,48 @@
 /*
 * A tool for deleting pads from the CLI, because sometimes a brick is required
 * to fix a window.
 */

 const request = require('../src/node_modules/request');
 const settings = require(`${__dirname}/../tests/container/loadSettings`).loadSettings();
 const supertest = require(`${__dirname}/../src/node_modules/supertest`);
 const api = supertest(`http://${settings.ip}:${settings.port}`);
 const path = require('path');
 const fs = require('fs');
 if (process.argv.length != 3) {
  console.error('Use: node deletePad.js $PADID');
  process.exit(1);
 }

 // get the padID
 const padId = process.argv[2];

 // get the API Key
 const filePath = path.join(__dirname, '../APIKEY.txt');
 const apikey = fs.readFileSync(filePath, {encoding: 'utf-8'});

 // Set apiVersion to base value, we change this later.
 let apiVersion = 1;

 // Update the apiVersion
 api.get('/api/')
    .expect((res) => {
      apiVersion = res.body.currentVersion;
      if (!res.body.currentVersion) throw new Error('No version set in API');
      return;
    })
    .end((err, res) => {
    // Now we know the latest API version, let's delete pad
      const uri = `/api/${apiVersion}/deletePad?apikey=${apikey}&padID=${padId}`;
      api.post(uri)
          .expect((res) => {
            if (res.body.code === 1) {
              console.error('Error deleting pad', res.body);
            } else {
              console.log('Deleted pad', res.body);
            }
            return;
          })
          .end(() => {});
    });
 // end
--- a/bin/dirty-db-cleaner.py
+++ b/bin/dirty-db-cleaner.py
@ -0,0 +1,48 @@
 #!/usr/bin/env PYTHONUNBUFFERED=1 python
 #
 # Created by Bjarni R. Einarsson, placed in the public domain. Go wild!
 #
 import json
 import os
 import sys

 try:
    dirtydb_input = sys.argv[1]
    dirtydb_output = '%s.new' % dirtydb_input
    assert(os.path.exists(dirtydb_input))
    assert(not os.path.exists(dirtydb_output))
 except:
    print()
    print('Usage: %s /path/to/dirty.db' % sys.argv[0])
    print()
    print('Note: Will create a file named dirty.db.new in the same folder,')
    print('      please make sure permissions are OK and a file by that')
    print('      name does not exist already. This script works by omitting')
    print('      duplicate lines from the dirty.db file, keeping only the')
    print('      last (latest) instance. No revision data should be lost,')
    print('      but be careful, make backups. If it breaks you get to keep')
    print('      both pieces!')
    print()
    sys.exit(1)

 dirtydb = {}
 lines = 0
 with open(dirtydb_input, 'r') as fd:
    print('Reading %s' % dirtydb_input)
    for line in fd:
        lines += 1
        try:
            data = json.loads(line)
            dirtydb[data['key']] = line
        except:
            print("Skipping invalid JSON!")
        if lines % 10000 == 0:
            sys.stderr.write('.')
 print()
 print('OK, found %d unique keys in %d lines' % (len(dirtydb), lines))

 with open(dirtydb_output, 'w') as fd:
    for data in list(dirtydb.values()):
        fd.write(data)

 print('Wrote data to %s. All done!' % dirtydb_output)
--- a/bin/doc/LICENSE
+++ b/bin/doc/LICENSE
@ -0,0 +1,18 @@
 Copyright Joyent, Inc. and other Node contributors. All rights reserved.
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to
 deal in the Software without restriction, including without limitation the
 rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
 sell copies of the Software, and to permit persons to whom the Software is
 furnished to do so, subject to the following conditions:

 The above copyright notice and this permission notice shall be included in
 all copies or substantial portions of the Software.

 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
 IN THE SOFTWARE.
--- a/bin/doc/README.md
+++ b/bin/doc/README.md
@ -0,0 +1,76 @@
 Here's how the node docs work.

 Each type of heading has a description block.


    ## module

        Stability: 3 - Stable

    description and examples.

    ### module.property

    * Type

    description of the property.

    ### module.someFunction(x, y, [z=100])

    * `x` {String} the description of the string
    * `y` {Boolean} Should I stay or should I go?
    * `z` {Number} How many zebras to bring.

    A description of the function.

    ### Event: 'blerg'

    * Argument: SomeClass object.

    Modules don't usually raise events on themselves.  `cluster` is the
    only exception.

    ## Class: SomeClass

    description of the class.

    ### Class Method: SomeClass.classMethod(anArg)

    * `anArg` {Object}  Just an argument
      * `field` {String} anArg can have this field.
      * `field2` {Boolean}  Another field.  Default: `false`.
    * Return: {Boolean} `true` if it worked.

    Description of the method for humans.

    ### someClass.nextSibling()

    * Return: {SomeClass object | null}  The next someClass in line.

    ### someClass.someProperty

    * String

    The indication of what someProperty is.

    ### Event: 'grelb'

    * `isBlerg` {Boolean}

    This event is emitted on instances of SomeClass, not on the module itself.


 * Modules have (description, Properties, Functions, Classes, Examples)
 * Properties have (type, description)
 * Functions have (list of arguments, description)
 * Classes have (description, Properties, Methods, Events)
 * Events have (list of arguments, description)
 * Methods have (list of arguments, description)
 * Properties have (type, description)

 # CLI usage

 Run the following from the etherpad-lite root directory:
 ```sh
 $ node bin/doc/generate doc/index.md --format=html --template=doc/template.html > out.html
 ```
--- a/bin/doc/generate.js
+++ b/bin/doc/generate.js
@ -0,0 +1,120 @@
 #!/usr/bin/env node
 // Copyright Joyent, Inc. and other Node contributors.
 //
 // Permission is hereby granted, free of charge, to any person obtaining a
 // copy of this software and associated documentation files (the
 // "Software"), to deal in the Software without restriction, including
 // without limitation the rights to use, copy, modify, merge, publish,
 // distribute, sublicense, and/or sell copies of the Software, and to permit
 // persons to whom the Software is furnished to do so, subject to the
 // following conditions:
 //
 // The above copyright notice and this permission notice shall be included
 // in all copies or substantial portions of the Software.
 //
 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
 // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 // MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
 // NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
 // DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
 // OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
 // USE OR OTHER DEALINGS IN THE SOFTWARE.

 const marked = require('marked');
 const fs = require('fs');
 const path = require('path');

 // parse the args.
 // Don't use nopt or whatever for this.  It's simple enough.

 const args = process.argv.slice(2);
 let format = 'json';
 let template = null;
 let inputFile = null;

 args.forEach((arg) => {
  if (!arg.match(/^\-\-/)) {
    inputFile = arg;
  } else if (arg.match(/^\-\-format=/)) {
    format = arg.replace(/^\-\-format=/, '');
  } else if (arg.match(/^\-\-template=/)) {
    template = arg.replace(/^\-\-template=/, '');
  }
 });


 if (!inputFile) {
  throw new Error('No input file specified');
 }


 console.error('Input file = %s', inputFile);
 fs.readFile(inputFile, 'utf8', (er, input) => {
  if (er) throw er;
  // process the input for @include lines
  processIncludes(inputFile, input, next);
 });


 const includeExpr = /^@include\s+([A-Za-z0-9-_\/]+)(?:\.)?([a-zA-Z]*)$/gmi;
 const includeData = {};
 function processIncludes(inputFile, input, cb) {
  const includes = input.match(includeExpr);
  if (includes === null) return cb(null, input);
  let errState = null;
  console.error(includes);
  let incCount = includes.length;
  if (incCount === 0) cb(null, input);

  includes.forEach((include) => {
    let fname = include.replace(/^@include\s+/, '');
    if (!fname.match(/\.md$/)) fname += '.md';

    if (includeData.hasOwnProperty(fname)) {
      input = input.split(include).join(includeData[fname]);
      incCount--;
      if (incCount === 0) {
        return cb(null, input);
      }
    }

    const fullFname = path.resolve(path.dirname(inputFile), fname);
    fs.readFile(fullFname, 'utf8', (er, inc) => {
      if (errState) return;
      if (er) return cb(errState = er);
      processIncludes(fullFname, inc, (er, inc) => {
        if (errState) return;
        if (er) return cb(errState = er);
        incCount--;
        includeData[fname] = inc;
        input = input.split(include).join(includeData[fname]);
        if (incCount === 0) {
          return cb(null, input);
        }
      });
    });
  });
 }


 function next(er, input) {
  if (er) throw er;
  switch (format) {
    case 'json':
      require('./json.js')(input, inputFile, (er, obj) => {
        console.log(JSON.stringify(obj, null, 2));
        if (er) throw er;
      });
      break;

    case 'html':
      require('./html.js')(input, inputFile, template, (er, html) => {
        if (er) throw er;
        console.log(html);
      });
      break;

    default:
      throw new Error(`Invalid format: ${format}`);
  }
 }
--- a/bin/doc/html.js
+++ b/bin/doc/html.js
@ -0,0 +1,172 @@
 // Copyright Joyent, Inc. and other Node contributors.
 //
 // Permission is hereby granted, free of charge, to any person obtaining a
 // copy of this software and associated documentation files (the
 // "Software"), to deal in the Software without restriction, including
 // without limitation the rights to use, copy, modify, merge, publish,
 // distribute, sublicense, and/or sell copies of the Software, and to permit
 // persons to whom the Software is furnished to do so, subject to the
 // following conditions:
 //
 // The above copyright notice and this permission notice shall be included
 // in all copies or substantial portions of the Software.
 //
 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
 // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 // MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
 // NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
 // DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
 // OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
 // USE OR OTHER DEALINGS IN THE SOFTWARE.

 const fs = require('fs');
 const marked = require('marked');
 const path = require('path');

 module.exports = toHTML;

 function toHTML(input, filename, template, cb) {
  const lexed = marked.lexer(input);
  fs.readFile(template, 'utf8', (er, template) => {
    if (er) return cb(er);
    render(lexed, filename, template, cb);
  });
 }

 function render(lexed, filename, template, cb) {
  // get the section
  const section = getSection(lexed);

  filename = path.basename(filename, '.md');

  lexed = parseLists(lexed);

  // generate the table of contents.
  // this mutates the lexed contents in-place.
  buildToc(lexed, filename, (er, toc) => {
    if (er) return cb(er);

    template = template.replace(/__FILENAME__/g, filename);
    template = template.replace(/__SECTION__/g, section);
    template = template.replace(/__TOC__/g, toc);

    // content has to be the last thing we do with
    // the lexed tokens, because it's destructive.
    content = marked.parser(lexed);
    template = template.replace(/__CONTENT__/g, content);

    cb(null, template);
  });
 }


 // just update the list item text in-place.
 // lists that come right after a heading are what we're after.
 function parseLists(input) {
  let state = null;
  let depth = 0;
  const output = [];
  output.links = input.links;
  input.forEach((tok) => {
    if (state === null) {
      if (tok.type === 'heading') {
        state = 'AFTERHEADING';
      }
      output.push(tok);
      return;
    }
    if (state === 'AFTERHEADING') {
      if (tok.type === 'list_start') {
        state = 'LIST';
        if (depth === 0) {
          output.push({type: 'html', text: '<div class="signature">'});
        }
        depth++;
        output.push(tok);
        return;
      }
      state = null;
      output.push(tok);
      return;
    }
    if (state === 'LIST') {
      if (tok.type === 'list_start') {
        depth++;
        output.push(tok);
        return;
      }
      if (tok.type === 'list_end') {
        depth--;
        if (depth === 0) {
          state = null;
          output.push({type: 'html', text: '</div>'});
        }
        output.push(tok);
        return;
      }
      if (tok.text) {
        tok.text = parseListItem(tok.text);
      }
    }
    output.push(tok);
  });

  return output;
 }


 function parseListItem(text) {
  text = text.replace(/\{([^\}]+)\}/, '<span class="type">$1</span>');
  // XXX maybe put more stuff here?
  return text;
 }


 // section is just the first heading
 function getSection(lexed) {
  const section = '';
  for (let i = 0, l = lexed.length; i < l; i++) {
    const tok = lexed[i];
    if (tok.type === 'heading') return tok.text;
  }
  return '';
 }


 function buildToc(lexed, filename, cb) {
  const indent = 0;
  let toc = [];
  let depth = 0;
  lexed.forEach((tok) => {
    if (tok.type !== 'heading') return;
    if (tok.depth - depth > 1) {
      return cb(new Error(`Inappropriate heading level\n${
        JSON.stringify(tok)}`));
    }

    depth = tok.depth;
    const id = getId(`${filename}_${tok.text.trim()}`);
    toc.push(`${new Array((depth - 1) * 2 + 1).join(' ')
    }* <a href="#${id}">${
      tok.text}</a>`);
    tok.text += `<span><a class="mark" href="#${id}" ` +
                `id="${id}">#</a></span>`;
  });

  toc = marked.parse(toc.join('\n'));
  cb(null, toc);
 }

 const idCounters = {};
 function getId(text) {
  text = text.toLowerCase();
  text = text.replace(/[^a-z0-9]+/g, '_');
  text = text.replace(/^_+|_+$/, '');
  text = text.replace(/^([^a-z])/, '_$1');
  if (idCounters.hasOwnProperty(text)) {
    text += `_${++idCounters[text]}`;
  } else {
    idCounters[text] = 0;
  }
  return text;
 }
--- a/bin/doc/json.js
+++ b/bin/doc/json.js
@ -0,0 +1,554 @@
 // Copyright Joyent, Inc. and other Node contributors.
 //
 // Permission is hereby granted, free of charge, to any person obtaining a
 // copy of this software and associated documentation files (the
 // "Software"), to deal in the Software without restriction, including
 // without limitation the rights to use, copy, modify, merge, publish,
 // distribute, sublicense, and/or sell copies of the Software, and to permit
 // persons to whom the Software is furnished to do so, subject to the
 // following conditions:
 //
 // The above copyright notice and this permission notice shall be included
 // in all copies or substantial portions of the Software.
 //
 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
 // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 // MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
 // NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
 // DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
 // OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
 // USE OR OTHER DEALINGS IN THE SOFTWARE.

 module.exports = doJSON;

 // Take the lexed input, and return a JSON-encoded object
 // A module looks like this: https://gist.github.com/1777387

 const marked = require('marked');

 function doJSON(input, filename, cb) {
  const root = {source: filename};
  const stack = [root];
  let depth = 0;
  let current = root;
  let state = null;
  const lexed = marked.lexer(input);
  lexed.forEach((tok) => {
    const type = tok.type;
    let text = tok.text;

    // <!-- type = module -->
    // This is for cases where the markdown semantic structure is lacking.
    if (type === 'paragraph' || type === 'html') {
      const metaExpr = /<!--([^=]+)=([^\-]+)-->\n*/g;
      text = text.replace(metaExpr, (_0, k, v) => {
        current[k.trim()] = v.trim();
        return '';
      });
      text = text.trim();
      if (!text) return;
    }

    if (type === 'heading' &&
        !text.trim().match(/^example/i)) {
      if (tok.depth - depth > 1) {
        return cb(new Error(`Inappropriate heading level\n${
          JSON.stringify(tok)}`));
      }

      // Sometimes we have two headings with a single
      // blob of description.  Treat as a clone.
      if (current &&
          state === 'AFTERHEADING' &&
          depth === tok.depth) {
        const clone = current;
        current = newSection(tok);
        current.clone = clone;
        // don't keep it around on the stack.
        stack.pop();
      } else {
        // if the level is greater than the current depth,
        // then it's a child, so we should just leave the stack
        // as it is.
        // However, if it's a sibling or higher, then it implies
        // the closure of the other sections that came before.
        // root is always considered the level=0 section,
        // and the lowest heading is 1, so this should always
        // result in having a valid parent node.
        let d = tok.depth;
        while (d <= depth) {
          finishSection(stack.pop(), stack[stack.length - 1]);
          d++;
        }
        current = newSection(tok);
      }

      depth = tok.depth;
      stack.push(current);
      state = 'AFTERHEADING';
      return;
    } // heading

    // Immediately after a heading, we can expect the following
    //
    // { type: 'code', text: 'Stability: ...' },
    //
    // a list: starting with list_start, ending with list_end,
    // maybe containing other nested lists in each item.
    //
    // If one of these isn't found, then anything that comes between
    // here and the next heading should be parsed as the desc.
    let stability;
    if (state === 'AFTERHEADING') {
      if (type === 'code' &&
          (stability = text.match(/^Stability: ([0-5])(?:\s*-\s*)?(.*)$/))) {
        current.stability = parseInt(stability[1], 10);
        current.stabilityText = stability[2].trim();
        return;
      } else if (type === 'list_start' && !tok.ordered) {
        state = 'AFTERHEADING_LIST';
        current.list = current.list || [];
        current.list.push(tok);
        current.list.level = 1;
      } else {
        current.desc = current.desc || [];
        if (!Array.isArray(current.desc)) {
          current.shortDesc = current.desc;
          current.desc = [];
        }
        current.desc.push(tok);
        state = 'DESC';
      }
      return;
    }

    if (state === 'AFTERHEADING_LIST') {
      current.list.push(tok);
      if (type === 'list_start') {
        current.list.level++;
      } else if (type === 'list_end') {
        current.list.level--;
      }
      if (current.list.level === 0) {
        state = 'AFTERHEADING';
        processList(current);
      }
      return;
    }

    current.desc = current.desc || [];
    current.desc.push(tok);
  });

  // finish any sections left open
  while (root !== (current = stack.pop())) {
    finishSection(current, stack[stack.length - 1]);
  }

  return cb(null, root);
 }


 // go from something like this:
 // [ { type: 'list_item_start' },
 //   { type: 'text',
 //     text: '`settings` Object, Optional' },
 //   { type: 'list_start', ordered: false },
 //   { type: 'list_item_start' },
 //   { type: 'text',
 //     text: 'exec: String, file path to worker file.  Default: `__filename`' },
 //   { type: 'list_item_end' },
 //   { type: 'list_item_start' },
 //   { type: 'text',
 //     text: 'args: Array, string arguments passed to worker.' },
 //   { type: 'text',
 //     text: 'Default: `process.argv.slice(2)`' },
 //   { type: 'list_item_end' },
 //   { type: 'list_item_start' },
 //   { type: 'text',
 //     text: 'silent: Boolean, whether or not to send output to parent\'s stdio.' },
 //   { type: 'text', text: 'Default: `false`' },
 //   { type: 'space' },
 //   { type: 'list_item_end' },
 //   { type: 'list_end' },
 //   { type: 'list_item_end' },
 //   { type: 'list_end' } ]
 // to something like:
 // [ { name: 'settings',
 //     type: 'object',
 //     optional: true,
 //     settings:
 //      [ { name: 'exec',
 //          type: 'string',
 //          desc: 'file path to worker file',
 //          default: '__filename' },
 //        { name: 'args',
 //          type: 'array',
 //          default: 'process.argv.slice(2)',
 //          desc: 'string arguments passed to worker.' },
 //        { name: 'silent',
 //          type: 'boolean',
 //          desc: 'whether or not to send output to parent\'s stdio.',
 //          default: 'false' } ] } ]

 function processList(section) {
  const list = section.list;
  const values = [];
  let current;
  const stack = [];

  // for now, *just* build the hierarchical list
  list.forEach((tok) => {
    const type = tok.type;
    if (type === 'space') return;
    if (type === 'list_item_start') {
      if (!current) {
        var n = {};
        values.push(n);
        current = n;
      } else {
        current.options = current.options || [];
        stack.push(current);
        var n = {};
        current.options.push(n);
        current = n;
      }
      return;
    } else if (type === 'list_item_end') {
      if (!current) {
        throw new Error(`invalid list - end without current item\n${
          JSON.stringify(tok)}\n${
          JSON.stringify(list)}`);
      }
      current = stack.pop();
    } else if (type === 'text') {
      if (!current) {
        throw new Error(`invalid list - text without current item\n${
          JSON.stringify(tok)}\n${
          JSON.stringify(list)}`);
      }
      current.textRaw = current.textRaw || '';
      current.textRaw += `${tok.text} `;
    }
  });

  // shove the name in there for properties, since they are always
  // just going to be the value etc.
  if (section.type === 'property' && values[0]) {
    values[0].textRaw = `\`${section.name}\` ${values[0].textRaw}`;
  }

  // now pull the actual values out of the text bits.
  values.forEach(parseListItem);

  // Now figure out what this list actually means.
  // depending on the section type, the list could be different things.

  switch (section.type) {
    case 'ctor':
    case 'classMethod':
    case 'method':
      // each item is an argument, unless the name is 'return',
      // in which case it's the return value.
      section.signatures = section.signatures || [];
      var sig = {};
      section.signatures.push(sig);
      sig.params = values.filter((v) => {
        if (v.name === 'return') {
          sig.return = v;
          return false;
        }
        return true;
      });
      parseSignature(section.textRaw, sig);
      break;

    case 'property':
      // there should be only one item, which is the value.
      // copy the data up to the section.
      var value = values[0] || {};
      delete value.name;
      section.typeof = value.type;
      delete value.type;
      Object.keys(value).forEach((k) => {
        section[k] = value[k];
      });
      break;

    case 'event':
      // event: each item is an argument.
      section.params = values;
      break;
  }

  // section.listParsed = values;
  delete section.list;
 }


 // textRaw = "someobject.someMethod(a, [b=100], [c])"
 function parseSignature(text, sig) {
  let params = text.match(paramExpr);
  if (!params) return;
  params = params[1];
  // the ] is irrelevant. [ indicates optionalness.
  params = params.replace(/\]/g, '');
  params = params.split(/,/);
  params.forEach((p, i, _) => {
    p = p.trim();
    if (!p) return;
    let param = sig.params[i];
    let optional = false;
    let def;
    // [foo] -> optional
    if (p.charAt(0) === '[') {
      optional = true;
      p = p.substr(1);
    }
    const eq = p.indexOf('=');
    if (eq !== -1) {
      def = p.substr(eq + 1);
      p = p.substr(0, eq);
    }
    if (!param) {
      param = sig.params[i] = {name: p};
    }
    // at this point, the name should match.
    if (p !== param.name) {
      console.error('Warning: invalid param "%s"', p);
      console.error(` > ${JSON.stringify(param)}`);
      console.error(` > ${text}`);
    }
    if (optional) param.optional = true;
    if (def !== undefined) param.default = def;
  });
 }


 function parseListItem(item) {
  if (item.options) item.options.forEach(parseListItem);
  if (!item.textRaw) return;

  // the goal here is to find the name, type, default, and optional.
  // anything left over is 'desc'
  let text = item.textRaw.trim();
  // text = text.replace(/^(Argument|Param)s?\s*:?\s*/i, '');

  text = text.replace(/^, /, '').trim();
  const retExpr = /^returns?\s*:?\s*/i;
  const ret = text.match(retExpr);
  if (ret) {
    item.name = 'return';
    text = text.replace(retExpr, '');
  } else {
    const nameExpr = /^['`"]?([^'`": \{]+)['`"]?\s*:?\s*/;
    const name = text.match(nameExpr);
    if (name) {
      item.name = name[1];
      text = text.replace(nameExpr, '');
    }
  }

  text = text.trim();
  const defaultExpr = / $Prime causes double exponent: use braces to clarify$ /i;
  const def = text.match(defaultExpr);
  if (def) {
    item.default = def[1];
    text = text.replace(defaultExpr, '');
  }

  text = text.trim();
  const typeExpr = /^\{([^\}]+)\}/;
  const type = text.match(typeExpr);
  if (type) {
    item.type = type[1];
    text = text.replace(typeExpr, '');
  }

  text = text.trim();
  const optExpr = /^Optional\.|(?:, )?Optional$/;
  const optional = text.match(optExpr);
  if (optional) {
    item.optional = true;
    text = text.replace(optExpr, '');
  }

  text = text.replace(/^\s*-\s*/, '');
  text = text.trim();
  if (text) item.desc = text;
 }


 function finishSection(section, parent) {
  if (!section || !parent) {
    throw new Error(`Invalid finishSection call\n${
      JSON.stringify(section)}\n${
      JSON.stringify(parent)}`);
  }

  if (!section.type) {
    section.type = 'module';
    if (parent && (parent.type === 'misc')) {
      section.type = 'misc';
    }
    section.displayName = section.name;
    section.name = section.name.toLowerCase()
        .trim().replace(/\s+/g, '_');
  }

  if (section.desc && Array.isArray(section.desc)) {
    section.desc.links = section.desc.links || [];
    section.desc = marked.parser(section.desc);
  }

  if (!section.list) section.list = [];
  processList(section);

  // classes sometimes have various 'ctor' children
  // which are actually just descriptions of a constructor
  // class signature.
  // Merge them into the parent.
  if (section.type === 'class' && section.ctors) {
    section.signatures = section.signatures || [];
    const sigs = section.signatures;
    section.ctors.forEach((ctor) => {
      ctor.signatures = ctor.signatures || [{}];
      ctor.signatures.forEach((sig) => {
        sig.desc = ctor.desc;
      });
      sigs.push.apply(sigs, ctor.signatures);
    });
    delete section.ctors;
  }

  // properties are a bit special.
  // their "type" is the type of object, not "property"
  if (section.properties) {
    section.properties.forEach((p) => {
      if (p.typeof) p.type = p.typeof;
      else delete p.type;
      delete p.typeof;
    });
  }

  // handle clones
  if (section.clone) {
    const clone = section.clone;
    delete section.clone;
    delete clone.clone;
    deepCopy(section, clone);
    finishSection(clone, parent);
  }

  let plur;
  if (section.type.slice(-1) === 's') {
    plur = `${section.type}es`;
  } else if (section.type.slice(-1) === 'y') {
    plur = section.type.replace(/y$/, 'ies');
  } else {
    plur = `${section.type}s`;
  }

  // if the parent's type is 'misc', then it's just a random
  // collection of stuff, like the "globals" section.
  // Make the children top-level items.
  if (section.type === 'misc') {
    Object.keys(section).forEach((k) => {
      switch (k) {
        case 'textRaw':
        case 'name':
        case 'type':
        case 'desc':
        case 'miscs':
          return;
        default:
          if (parent.type === 'misc') {
            return;
          }
          if (Array.isArray(k) && parent[k]) {
            parent[k] = parent[k].concat(section[k]);
          } else if (!parent[k]) {
            parent[k] = section[k];
          } else {
            // parent already has, and it's not an array.
            return;
          }
      }
    });
  }

  parent[plur] = parent[plur] || [];
  parent[plur].push(section);
 }


 // Not a general purpose deep copy.
 // But sufficient for these basic things.
 function deepCopy(src, dest) {
  Object.keys(src).filter((k) => !dest.hasOwnProperty(k)).forEach((k) => {
    dest[k] = deepCopy_(src[k]);
  });
 }

 function deepCopy_(src) {
  if (!src) return src;
  if (Array.isArray(src)) {
    var c = new Array(src.length);
    src.forEach((v, i) => {
      c[i] = deepCopy_(v);
    });
    return c;
  }
  if (typeof src === 'object') {
    var c = {};
    Object.keys(src).forEach((k) => {
      c[k] = deepCopy_(src[k]);
    });
    return c;
  }
  return src;
 }


 // these parse out the contents of an H# tag
 const eventExpr = /^Event(?::|\s)+['"]?([^"']+).*$/i;
 const classExpr = /^Class:\s*([^ ]+).*?$/i;
 const propExpr = /^(?:property:?\s*)?[^\.]+\.([^ \.]+)\s*?$/i;
 const braceExpr = /^(?:property:?\s*)?[^\. $Missing superscript or subscript argument$ ]+\])\s*?$/i;
 const classMethExpr =
  /^class\s*method\s*:?[^\.]+\.([^ \.]+) $Missing superscript or subscript argument$ ]*\)\s*?$/i;
 const methExpr =
  /^(?:method:?\s*)?(?:[^\.]+\.)?([^ \.]+) $Missing superscript or subscript argument$ ]*\)\s*?$/i;
 const newExpr = /^new ([A-Z][a-z]+) $Missing superscript or subscript argument$ ]*\)\s*?$/;
 var paramExpr = / $(. *)$ ;?$/;

 function newSection(tok) {
  const section = {};
  // infer the type from the text.
  const text = section.textRaw = tok.text;
  if (text.match(eventExpr)) {
    section.type = 'event';
    section.name = text.replace(eventExpr, '$1');
  } else if (text.match(classExpr)) {
    section.type = 'class';
    section.name = text.replace(classExpr, '$1');
  } else if (text.match(braceExpr)) {
    section.type = 'property';
    section.name = text.replace(braceExpr, '$1');
  } else if (text.match(propExpr)) {
    section.type = 'property';
    section.name = text.replace(propExpr, '$1');
  } else if (text.match(classMethExpr)) {
    section.type = 'classMethod';
    section.name = text.replace(classMethExpr, '$1');
  } else if (text.match(methExpr)) {
    section.type = 'method';
    section.name = text.replace(methExpr, '$1');
  } else if (text.match(newExpr)) {
    section.type = 'ctor';
    section.name = text.replace(newExpr, '$1');
  } else {
    section.name = text;
  }
  return section;
 }
--- a/bin/doc/package-lock.json
+++ b/bin/doc/package-lock.json
@ -0,0 +1,13 @@
 {
  "name": "node-doc-generator",
  "version": "0.0.0",
  "lockfileVersion": 1,
  "requires": true,
  "dependencies": {
    "marked": {
      "version": "0.8.2",
      "resolved": "https://registry.npmjs.org/marked/-/marked-0.8.2.tgz",
      "integrity": "sha512-EGwzEeCcLniFX51DhTpmTom+dSA/MG/OBUDjnWtHbEnjAH180VzUeAw+oE4+Zv+CoYBWyRlYOTR0N8SO9R1PVw=="
    }
  }
 }
--- a/bin/doc/package.json
+++ b/bin/doc/package.json
@ -0,0 +1,15 @@
 {
  "author": "Isaac Z. Schlueter <i@izs.me> (http://blog.izs.me/)",
  "name": "node-doc-generator",
  "description": "Internal tool for generating Node.js API docs",
  "version": "0.0.0",
  "engines": {
    "node": ">=0.6.10"
  },
  "dependencies": {
    "marked": "0.8.2"
  },
  "devDependencies": {},
  "optionalDependencies": {},
  "bin": "./generate.js"
 }
--- a/bin/extractPadData.js
+++ b/bin/extractPadData.js
@ -0,0 +1,75 @@
 /*
 * This is a debug tool. It helps to extract all datas of a pad and move it from
 * a productive environment and to a develop environment to reproduce bugs
 * there. It outputs a dirtydb file
 */

 if (process.argv.length != 3) {
  console.error('Use: node extractPadData.js $PADID');
  process.exit(1);
 }

 // get the padID
 const padId = process.argv[2];

 const npm = require('../src/node_modules/npm');

 npm.load({}, async (er) => {
  if (er) {
    console.error(`Could not load NPM: ${er}`);
    process.exit(1);
  }

  try {
    // initialize database
    const settings = require('../src/node/utils/Settings');
    const db = require('../src/node/db/DB');
    await db.init();

    // load extra modules
    const dirtyDB = require('../src/node_modules/dirty');
    const padManager = require('../src/node/db/PadManager');
    const util = require('util');

    // initialize output database
    const dirty = dirtyDB(`${padId}.db`);

    // Promise wrapped get and set function
    const wrapped = db.db.db.wrappedDB;
    const get = util.promisify(wrapped.get.bind(wrapped));
    const set = util.promisify(dirty.set.bind(dirty));

    // array in which required key values will be accumulated
    const neededDBValues = [`pad:${padId}`];

    // get the actual pad object
    const pad = await padManager.getPad(padId);

    // add all authors
    neededDBValues.push(...pad.getAllAuthors().map((author) => `globalAuthor:${author}`));

    // add all revisions
    for (let rev = 0; rev <= pad.head; ++rev) {
      neededDBValues.push(`pad:${padId}:revs:${rev}`);
    }

    // add all chat values
    for (let chat = 0; chat <= pad.chatHead; ++chat) {
      neededDBValues.push(`pad:${padId}:chat:${chat}`);
    }

    for (const dbkey of neededDBValues) {
      let dbvalue = await get(dbkey);
      if (dbvalue && typeof dbvalue !== 'object') {
        dbvalue = JSON.parse(dbvalue);
      }
      await set(dbkey, dbvalue);
    }

    console.log('finished');
    process.exit(0);
  } catch (er) {
    console.error(er);
    process.exit(1);
  }
 });
--- a/bin/fastRun.sh
+++ b/bin/fastRun.sh
@ -0,0 +1,25 @@
 #!/bin/bash
 #
 # Run Etherpad directly, assuming all the dependencies are already installed.
 #
 # Useful for developers, or users that know what they are doing. If you just
 # upgraded Etherpad version, installed a new dependency, or are simply unsure
 # of what to do, please execute bin/installDeps.sh once before running this
 # script.

 set -eu

 # source: https://stackoverflow.com/questions/59895/how-to-get-the-source-directory-of-a-bash-script-from-within-the-script-itself#246128
 DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"

 # Source constants and usefull functions
 . ${DIR}/../bin/functions.sh

 echo "Running directly, without checking/installing dependencies"

 # move to the base Etherpad directory. This will be necessary until Etherpad
 # learns to run from arbitrary CWDs.
 cd "${DIR}/.."

 # run Etherpad main class
 node $(compute_node_args) "${DIR}/../node_modules/ep_etherpad-lite/node/server.js" "$@"
--- a/bin/functions.sh
+++ b/bin/functions.sh
@ -0,0 +1,74 @@
 # minimum required node version
 REQUIRED_NODE_MAJOR=10
 REQUIRED_NODE_MINOR=13

 # minimum required npm version
 REQUIRED_NPM_MAJOR=5
 REQUIRED_NPM_MINOR=5

 pecho() { printf %s\n "$*"; }
 log() { pecho "$@"; }
 error() { log "ERROR: $@" >&2; }
 fatal() { error "$@"; exit 1; }
 is_cmd() { command -v "$@" >/dev/null 2>&1; }


 get_program_version() {
  PROGRAM="$1"
  KIND="${2:-full}"
  PROGRAM_VERSION_STRING=$($PROGRAM --version)
  PROGRAM_VERSION_STRING=${PROGRAM_VERSION_STRING#"v"}

  DETECTED_MAJOR=$(pecho "$PROGRAM_VERSION_STRING" | cut -s -d "." -f 1)
  [ -n "$DETECTED_MAJOR" ] || fatal "Cannot extract $PROGRAM major version from version string \"$PROGRAM_VERSION_STRING\""
  case "$DETECTED_MAJOR" in
      ''|*[!0-9]*)
        fatal "$PROGRAM_LABEL major version from \"$VERSION_STRING\" is not a number. Detected: \"$DETECTED_MAJOR\""
        ;;
  esac

  DETECTED_MINOR=$(pecho "$PROGRAM_VERSION_STRING" | cut -s -d "." -f 2)
  [ -n "$DETECTED_MINOR" ] || fatal "Cannot extract $PROGRAM minor version from version string \"$PROGRAM_VERSION_STRING\""
  case "$DETECTED_MINOR" in
      ''|*[!0-9]*)
        fatal "$PROGRAM_LABEL minor version from \"$VERSION_STRING\" is not a number. Detected: \"$DETECTED_MINOR\""
  esac

  case $KIND in
    major)
      echo $DETECTED_MAJOR
      exit;;
    minor)
      echo $DETECTED_MINOR
      exit;;
    *)
      echo $DETECTED_MAJOR.$DETECTED_MINOR
      exit;;
  esac

  echo $VERSION
 }


 compute_node_args() {
  ARGS=""

  NODE_MAJOR=$(get_program_version "node" "major")
  [ "$NODE_MAJOR" -eq "10" ] && ARGS="$ARGS --experimental-worker"

  echo $ARGS
 }


 require_minimal_version() {
  PROGRAM_LABEL="$1"
  VERSION="$2"
  REQUIRED_MAJOR="$3"
  REQUIRED_MINOR="$4"

  VERSION_MAJOR=$(pecho "$VERSION" | cut -s -d "." -f 1)
  VERSION_MINOR=$(pecho "$VERSION" | cut -s -d "." -f 2)

  [ "$VERSION_MAJOR" -gt "$REQUIRED_MAJOR" ] || ([ "$VERSION_MAJOR" -eq "$REQUIRED_MAJOR" ] && [ "$VERSION_MINOR" -ge "$REQUIRED_MINOR" ]) \
    || fatal "Your $PROGRAM_LABEL version \"$VERSION_MAJOR.$VERSION_MINOR\" is too old. $PROGRAM_LABEL $REQUIRED_MAJOR.$REQUIRED_MINOR.x or higher is required."
 }
--- a/bin/importSqlFile.js
+++ b/bin/importSqlFile.js
@ -0,0 +1,103 @@
 const startTime = Date.now();

 require('ep_etherpad-lite/node_modules/npm').load({}, (er, npm) => {
  const fs = require('fs');

  const ueberDB = require('ep_etherpad-lite/node_modules/ueberdb2');
  const settings = require('ep_etherpad-lite/node/utils/Settings');
  const log4js = require('ep_etherpad-lite/node_modules/log4js');

  const dbWrapperSettings = {
    cache: 0,
    writeInterval: 100,
    json: false, // data is already json encoded
  };
  const db = new ueberDB.database(settings.dbType, settings.dbSettings, dbWrapperSettings, log4js.getLogger('ueberDB'));

  const sqlFile = process.argv[2];

  // stop if the settings file is not set
  if (!sqlFile) {
    console.error('Use: node importSqlFile.js $SQLFILE');
    process.exit(1);
  }

  log('initializing db');
  db.init((err) => {
    // there was an error while initializing the database, output it and stop
    if (err) {
      console.error('ERROR: Problem while initializing the database');
      console.error(err.stack ? err.stack : err);
      process.exit(1);
    } else {
      log('done');

      log('open output file...');
      const lines = fs.readFileSync(sqlFile, 'utf8').split('\n');

      const count = lines.length;
      let keyNo = 0;

      process.stdout.write(`Start importing ${count} keys...\n`);
      lines.forEach((l) => {
        if (l.substr(0, 27) == 'REPLACE INTO store VALUES (') {
          const pos = l.indexOf("', '");
          const key = l.substr(28, pos - 28);
          let value = l.substr(pos + 3);
          value = value.substr(0, value.length - 2);
          console.log(`key: ${key} val: ${value}`);
          console.log(`unval: ${unescape(value)}`);
          db.set(key, unescape(value), null);
          keyNo++;
          if (keyNo % 1000 == 0) {
            process.stdout.write(` ${keyNo}/${count}\n`);
          }
        }
      });
      process.stdout.write('\n');
      process.stdout.write('done. waiting for db to finish transaction. depended on dbms this may take some time...\n');

      db.doShutdown(() => {
        log(`finished, imported ${keyNo} keys.`);
        process.exit(0);
      });
    }
  });
 });

 function log(str) {
  console.log(`${(Date.now() - startTime) / 1000}\t${str}`);
 }

 unescape = function (val) {
  // value is a string
  if (val.substr(0, 1) == "'") {
    val = val.substr(0, val.length - 1).substr(1);

    return val.replace(/\[0nrbtZ\'"]/g, (s) => {
      switch (s) {
        case '\0': return '\0';
        case '\n': return '\n';
        case '\r': return '\r';
        case '\b': return '\b';
        case '\t': return '\t';
        case '\Z': return '\x1a';
        default: return s.substr(1);
      }
    });
  }

  // value is a boolean or NULL
  if (val == 'NULL') {
    return null;
  }
  if (val == 'true') {
    return true;
  }
  if (val == 'false') {
    return false;
  }

  // value is a number
  return val;
 };
--- a/bin/installDeps.sh
+++ b/bin/installDeps.sh
@ -0,0 +1,52 @@
 #!/bin/sh

 # Move to the folder where ep-lite is installed
 cd "$(dirname "$0")"/..

 # Source constants and usefull functions
 . bin/functions.sh

 # Is node installed?
 # Not checking io.js, default installation creates a symbolic link to node
 is_cmd node || fatal "Please install node.js ( https://nodejs.org )"

 # Is npm installed?
 is_cmd npm || fatal "Please install npm ( https://npmjs.org )"

 # Check npm version
 require_minimal_version "npm" $(get_program_version "npm") "$REQUIRED_NPM_MAJOR" "$REQUIRED_NPM_MINOR"

 # Check node version
 require_minimal_version "nodejs" $(get_program_version "node") "$REQUIRED_NODE_MAJOR" "$REQUIRED_NODE_MINOR"

 # Get the name of the settings file
 settings="settings.json"
 a='';
 for arg in "$@"; do
  if [ "$a" = "--settings" ] || [ "$a" = "-s" ]; then settings=$arg; fi
  a=$arg
 done

 # Does a $settings exist? if not copy the template
 if [ ! -f "$settings" ]; then
  log "Copy the settings template to $settings..."
  cp settings.json.template "$settings" || exit 1
 fi

 log "Ensure that all dependencies are up to date...  If this is the first time you have run Etherpad please be patient."
 (
  mkdir -p node_modules
  cd node_modules
  [ -e ep_etherpad-lite ] || ln -s ../src ep_etherpad-lite
  cd ep_etherpad-lite
  npm ci
 ) || {
  rm -rf src/node_modules
  exit 1
 }

 # Remove all minified data to force node creating it new
 log "Clearing minified cache..."
 rm -f var/minified*

 exit 0
--- a/bin/installOnWindows.bat
+++ b/bin/installOnWindows.bat
@ -0,0 +1,34 @@
@echo off

 :: Change directory to etherpad-lite root
 cd /D "%~dp0\.."

 :: Is node installed?
 cmd /C node -e "" || ( echo "Please install node.js ( https://nodejs.org )" && exit /B 1 )

 echo _
 echo Ensure that all dependencies are up to date...  If this is the first time you have run Etherpad please be patient.

 mkdir node_modules
 cd /D node_modules
 mklink /D "ep_etherpad-lite" "..\src"

 cd /D "ep_etherpad-lite"
 cmd /C npm ci || exit /B 1

 cd /D "%~dp0\.."

 echo _
 echo Clearing cache...
 del /S var\minified*

 echo _
 echo Setting up settings.json...
 IF NOT EXIST settings.json (
  echo Can't find settings.json.
  echo Copying settings.json.template...
  cmd /C copy settings.json.template settings.json || exit /B 1
 )

 echo _
 echo Installed Etherpad!  To run Etherpad type start.bat
--- a/bin/migrateDirtyDBtoRealDB.js
+++ b/bin/migrateDirtyDBtoRealDB.js
@ -0,0 +1,48 @@
 require('ep_etherpad-lite/node_modules/npm').load({}, (er, npm) => {
  process.chdir(`${npm.root}/..`);

  // This script requires that you have modified your settings.json file
  // to work with a real database.  Please make a backup of your dirty.db
  // file before using this script, just to be safe.

  // It might be necessary to run the script using more memory:
  // `node --max-old-space-size=4096 bin/migrateDirtyDBtoRealDB.js`


  const settings = require('ep_etherpad-lite/node/utils/Settings');
  let dirty = require('../src/node_modules/dirty');
  const ueberDB = require('../src/node_modules/ueberdb2');
  const log4js = require('../src/node_modules/log4js');
  const dbWrapperSettings = {
    cache: '0', // The cache slows things down when you're mostly writing.
    writeInterval: 0, // Write directly to the database, don't buffer
  };
  const db = new ueberDB.database(settings.dbType, settings.dbSettings, dbWrapperSettings, log4js.getLogger('ueberDB'));
  let i = 0;
  let length = 0;

  db.init(() => {
    console.log('Waiting for dirtyDB to parse its file.');
    dirty = dirty('var/dirty.db').on('load', () => {
      dirty.forEach(() => {
        length++;
      });
      console.log(`Found ${length} records, processing now.`);

      dirty.forEach(async (key, value) => {
        const error = await db.set(key, value);
        console.log(`Wrote record ${i}`);
        i++;

        if (i === length) {
          console.log('finished, just clearing up for a bit...');
          setTimeout(() => {
            process.exit(0);
          }, 5000);
        }
      });
      console.log('Please wait for all records to flush to database, then kill this process.');
    });
    console.log('done?');
  });
 });
--- a/bin/plugins/README.md
+++ b/bin/plugins/README.md
@ -0,0 +1,52 @@
 The files in this folder are for Plugin developers.

 # Get suggestions to improve your Plugin

 This code will check your plugin for known usual issues and some suggestions for improvements.  No changes will be made to your project.

 ```
 node bin/plugins/checkPlugin.js  $P L U G I N_{N} A M E$ 
 ```

 # Basic Example:
 ```
 node bin/plugins/checkPlugin.js ep_webrtc
 ```

 ## Autofixing - will autofix any issues it can
 ```
 node bin/plugins/checkPlugins.js ep_whatever autofix
 ```

 ## Autocommitting, push, npm minor patch and npm publish (highly dangerous)
 ```
 node bin/plugins/checkPlugins.js ep_whatever autofix autocommit
 ```

 # All the plugins
 Replace johnmclear with your github username

 ```
 # Clones
 cd node_modules
 GHUSER=johnmclear; curl "https://api.github.com/users/$GHUSER/repos?per_page=1000" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
 cd ..

 # autofixes and autocommits /pushes & npm publishes
 for dir in `ls node_modules`;
 do
 # echo $0
 if [[ $dir == *"ep_"* ]]; then
 if [[ $dir != "ep_etherpad-lite" ]]; then
 node bin/plugins/checkPlugin.js $dir autofix autocommit
 fi
 fi
 # echo $dir
 done
 ```

 # Automating update of ether organization plugins
 ```
 getCorePlugins.sh
 updateCorePlugins.sh
 ```
--- a/bin/plugins/checkPlugin.js
+++ b/bin/plugins/checkPlugin.js
@ -0,0 +1,469 @@
 /*
 *
 * Usage -- see README.md
 *
 * Normal usage:                node bin/plugins/checkPlugins.js ep_whatever
 * Auto fix the things it can:  node bin/plugins/checkPlugins.js ep_whatever autofix
 * Auto commit, push and publish(to npm) * highly dangerous:
 node bin/plugins/checkPlugins.js ep_whatever autofix autocommit

 */

 const fs = require('fs');
 const {exec} = require('child_process');

 // get plugin name & path from user input
 const pluginName = process.argv[2];

 if (!pluginName) {
  console.error('no plugin name specified');
  process.exit(1);
 }

 const pluginPath = `node_modules/${pluginName}`;

 console.log(`Checking the plugin: ${pluginName}`);

 // Should we autofix?
 if (process.argv[3] && process.argv[3] === 'autofix') var autoFix = true;

 // Should we update files where possible?
 if (process.argv[5] && process.argv[5] === 'autoupdate') var autoUpdate = true;

 // Should we automcommit and npm publish?!
 if (process.argv[4] && process.argv[4] === 'autocommit') var autoCommit = true;


 if (autoCommit) {
  console.warn('Auto commit is enabled, I hope you know what you are doing...');
 }

 fs.readdir(pluginPath, (err, rootFiles) => {
  // handling error
  if (err) {
    return console.log(`Unable to scan directory: ${err}`);
  }

  // rewriting files to lower case
  const files = [];

  // some files we need to know the actual file name.  Not compulsory but might help in the future.
  let readMeFileName;
  let repository;
  let hasAutoFixed = false;

  for (let i = 0; i < rootFiles.length; i++) {
    if (rootFiles[i].toLowerCase().indexOf('readme') !== -1) readMeFileName = rootFiles[i];
    files.push(rootFiles[i].toLowerCase());
  }

  if (files.indexOf('.git') === -1) {
    console.error('No .git folder, aborting');
    process.exit(1);
  }

  // do a git pull...
  var child_process = require('child_process');
  try {
    child_process.execSync('git pull ', {cwd: `${pluginPath}/`});
  } catch (e) {
    console.error('Error git pull', e);
  }

  try {
    const path = `${pluginPath}/.github/workflows/npmpublish.yml`;
    if (!fs.existsSync(path)) {
      console.log('no .github/workflows/npmpublish.yml, create one and set npm secret to auto publish to npm on commit');
      if (autoFix) {
        const npmpublish =
            fs.readFileSync('bin/plugins/lib/npmpublish.yml', {encoding: 'utf8', flag: 'r'});
        fs.mkdirSync(`${pluginPath}/.github/workflows`, {recursive: true});
        fs.writeFileSync(path, npmpublish);
        hasAutoFixed = true;
        console.log("If you haven't already, setup autopublish for this plugin https://github.com/ether/etherpad-lite/wiki/Plugins:-Automatically-publishing-to-npm-on-commit-to-Github-Repo");
      } else {
        console.log('Setup autopublish for this plugin https://github.com/ether/etherpad-lite/wiki/Plugins:-Automatically-publishing-to-npm-on-commit-to-Github-Repo');
      }
    } else {
      // autopublish exists, we should check the version..
      // checkVersion takes two file paths and checks for a version string in them.
      const currVersionFile = fs.readFileSync(path, {encoding: 'utf8', flag: 'r'});
      const existingConfigLocation = currVersionFile.indexOf('##ETHERPAD_NPM_V=');
      const existingValue = parseInt(currVersionFile.substr(existingConfigLocation + 17, existingConfigLocation.length));

      const reqVersionFile = fs.readFileSync('bin/plugins/lib/npmpublish.yml', {encoding: 'utf8', flag: 'r'});
      const reqConfigLocation = reqVersionFile.indexOf('##ETHERPAD_NPM_V=');
      const reqValue = parseInt(reqVersionFile.substr(reqConfigLocation + 17, reqConfigLocation.length));

      if (!existingValue || (reqValue > existingValue)) {
        const npmpublish =
            fs.readFileSync('bin/plugins/lib/npmpublish.yml', {encoding: 'utf8', flag: 'r'});
        fs.mkdirSync(`${pluginPath}/.github/workflows`, {recursive: true});
        fs.writeFileSync(path, npmpublish);
        hasAutoFixed = true;
      }
    }
  } catch (err) {
    console.error(err);
  }


  try {
    const path = `${pluginPath}/.github/workflows/backend-tests.yml`;
    if (!fs.existsSync(path)) {
      console.log('no .github/workflows/backend-tests.yml, create one and set npm secret to auto publish to npm on commit');
      if (autoFix) {
        const backendTests =
            fs.readFileSync('bin/plugins/lib/backend-tests.yml', {encoding: 'utf8', flag: 'r'});
        fs.mkdirSync(`${pluginPath}/.github/workflows`, {recursive: true});
        fs.writeFileSync(path, backendTests);
        hasAutoFixed = true;
      }
    } else {
      // autopublish exists, we should check the version..
      // checkVersion takes two file paths and checks for a version string in them.
      const currVersionFile = fs.readFileSync(path, {encoding: 'utf8', flag: 'r'});
      const existingConfigLocation = currVersionFile.indexOf('##ETHERPAD_NPM_V=');
      const existingValue = parseInt(currVersionFile.substr(existingConfigLocation + 17, existingConfigLocation.length));

      const reqVersionFile = fs.readFileSync('bin/plugins/lib/backend-tests.yml', {encoding: 'utf8', flag: 'r'});
      const reqConfigLocation = reqVersionFile.indexOf('##ETHERPAD_NPM_V=');
      const reqValue = parseInt(reqVersionFile.substr(reqConfigLocation + 17, reqConfigLocation.length));

      if (!existingValue || (reqValue > existingValue)) {
        const backendTests =
            fs.readFileSync('bin/plugins/lib/backend-tests.yml', {encoding: 'utf8', flag: 'r'});
        fs.mkdirSync(`${pluginPath}/.github/workflows`, {recursive: true});
        fs.writeFileSync(path, backendTests);
        hasAutoFixed = true;
      }
    }
  } catch (err) {
    console.error(err);
  }

  if (files.indexOf('package.json') === -1) {
    console.warn('no package.json, please create');
  }

  if (files.indexOf('package.json') !== -1) {
    const packageJSON = fs.readFileSync(`${pluginPath}/package.json`, {encoding: 'utf8', flag: 'r'});
    const parsedPackageJSON = JSON.parse(packageJSON);
    if (autoFix) {
      let updatedPackageJSON = false;
      if (!parsedPackageJSON.funding) {
        updatedPackageJSON = true;
        parsedPackageJSON.funding = {
          type: 'individual',
          url: 'https://etherpad.org/',
        };
      }
      if (updatedPackageJSON) {
        hasAutoFixed = true;
        fs.writeFileSync(`${pluginPath}/package.json`, JSON.stringify(parsedPackageJSON, null, 2));
      }
    }

    if (packageJSON.toLowerCase().indexOf('repository') === -1) {
      console.warn('No repository in package.json');
      if (autoFix) {
        console.warn('Repository not detected in package.json.  Please add repository section manually.');
      }
    } else {
      // useful for creating README later.
      repository = parsedPackageJSON.repository.url;
    }

    // include lint config
    if (packageJSON.toLowerCase().indexOf('devdependencies') === -1 || !parsedPackageJSON.devDependencies.eslint) {
      console.warn('Missing eslint reference in devDependencies');
      if (autoFix) {
        const devDependencies = {
          'eslint': '^7.14.0',
          'eslint-config-etherpad': '^1.0.13',
          'eslint-plugin-mocha': '^8.0.0',
          'eslint-plugin-node': '^11.1.0',
          'eslint-plugin-prefer-arrow': '^1.2.2',
          'eslint-plugin-promise': '^4.2.1',
        };
        hasAutoFixed = true;
        parsedPackageJSON.devDependencies = devDependencies;
        fs.writeFileSync(`${pluginPath}/package.json`, JSON.stringify(parsedPackageJSON, null, 2));

        const child_process = require('child_process');
        try {
          child_process.execSync('npm install', {cwd: `${pluginPath}/`});
          hasAutoFixed = true;
        } catch (e) {
          console.error('Failed to create package-lock.json');
        }
      }
    }

    // include peer deps config
    if (packageJSON.toLowerCase().indexOf('peerdependencies') === -1 || !parsedPackageJSON.peerDependencies) {
      console.warn('Missing peer deps reference in package.json');
      if (autoFix) {
        const peerDependencies = {
          'ep_etherpad-lite': '>=1.8.6',
        };
        hasAutoFixed = true;
        parsedPackageJSON.peerDependencies = peerDependencies;
        fs.writeFileSync(`${pluginPath}/package.json`, JSON.stringify(parsedPackageJSON, null, 2));
        const child_process = require('child_process');
        try {
          child_process.execSync('npm install --no-save ep_etherpad-lite@file:../../src', {cwd: `${pluginPath}/`});
          hasAutoFixed = true;
        } catch (e) {
          console.error('Failed to create package-lock.json');
        }
      }
    }

    if (packageJSON.toLowerCase().indexOf('eslintconfig') === -1) {
      console.warn('No esLintConfig in package.json');
      if (autoFix) {
        const eslintConfig = {
          root: true,
          extends: 'etherpad/plugin',
        };
        hasAutoFixed = true;
        parsedPackageJSON.eslintConfig = eslintConfig;
        fs.writeFileSync(`${pluginPath}/package.json`, JSON.stringify(parsedPackageJSON, null, 2));
      }
    }

    if (packageJSON.toLowerCase().indexOf('scripts') === -1) {
      console.warn('No scripts in package.json');
      if (autoFix) {
        const scripts = {
          'lint': 'eslint .',
          'lint:fix': 'eslint --fix .',
        };
        hasAutoFixed = true;
        parsedPackageJSON.scripts = scripts;
        fs.writeFileSync(`${pluginPath}/package.json`, JSON.stringify(parsedPackageJSON, null, 2));
      }
    }

    if ((packageJSON.toLowerCase().indexOf('engines') === -1) || !parsedPackageJSON.engines.node) {
      console.warn('No engines or node engine in package.json');
      if (autoFix) {
        const engines = {
          node: '>=10.13.0',
        };
        hasAutoFixed = true;
        parsedPackageJSON.engines = engines;
        fs.writeFileSync(`${pluginPath}/package.json`, JSON.stringify(parsedPackageJSON, null, 2));
      }
    }
  }

  if (files.indexOf('package-lock.json') === -1) {
    console.warn('package-lock.json file not found.  Please run npm install in the plugin folder and commit the package-lock.json file.');
    if (autoFix) {
      var child_process = require('child_process');
      try {
        child_process.execSync('npm install', {cwd: `${pluginPath}/`});
        console.log('Making package-lock.json');
        hasAutoFixed = true;
      } catch (e) {
        console.error('Failed to create package-lock.json');
      }
    }
  }

  if (files.indexOf('readme') === -1 && files.indexOf('readme.md') === -1) {
    console.warn('README.md file not found, please create');
    if (autoFix) {
      console.log('Autofixing missing README.md file, please edit the README.md file further to include plugin specific details.');
      let readme = fs.readFileSync('bin/plugins/lib/README.md', {encoding: 'utf8', flag: 'r'});
      readme = readme.replace(/ $p l u g i n_{n} a m e$ /g, pluginName);
      if (repository) {
        const org = repository.split('/')[3];
        const name = repository.split('/')[4];
        readme = readme.replace(/ $o r g_{n} a m e$ /g, org);
        readme = readme.replace(/ $r e p o_{u} r l$ /g, name);
        fs.writeFileSync(`${pluginPath}/README.md`, readme);
      } else {
        console.warn('Unable to find repository in package.json, aborting.');
      }
    }
  }

  if (files.indexOf('contributing') === -1 && files.indexOf('contributing.md') === -1) {
    console.warn('CONTRIBUTING.md file not found, please create');
    if (autoFix) {
      console.log('Autofixing missing CONTRIBUTING.md file, please edit the CONTRIBUTING.md file further to include plugin specific details.');
      let contributing = fs.readFileSync('bin/plugins/lib/CONTRIBUTING.md', {encoding: 'utf8', flag: 'r'});
      contributing = contributing.replace(/ $p l u g i n_{n} a m e$ /g, pluginName);
      fs.writeFileSync(`${pluginPath}/CONTRIBUTING.md`, contributing);
    }
  }


  if (files.indexOf('readme') !== -1 && files.indexOf('readme.md') !== -1) {
    const readme = fs.readFileSync(`${pluginPath}/${readMeFileName}`, {encoding: 'utf8', flag: 'r'});
    if (readme.toLowerCase().indexOf('license') === -1) {
      console.warn('No license section in README');
      if (autoFix) {
        console.warn('Please add License section to README manually.');
      }
    }
  }

  if (files.indexOf('license') === -1 && files.indexOf('license.md') === -1) {
    console.warn('LICENSE.md file not found, please create');
    if (autoFix) {
      hasAutoFixed = true;
      console.log('Autofixing missing LICENSE.md file, including Apache 2 license.');
      exec('git config user.name', (error, name, stderr) => {
        if (error) {
          console.log(`error: ${error.message}`);
          return;
        }
        if (stderr) {
          console.log(`stderr: ${stderr}`);
          return;
        }
        let license = fs.readFileSync('bin/plugins/lib/LICENSE.md', {encoding: 'utf8', flag: 'r'});
        license = license.replace('[yyyy]', new Date().getFullYear());
        license = license.replace('[name of copyright owner]', name);
        fs.writeFileSync(`${pluginPath}/LICENSE.md`, license);
      });
    }
  }

  let travisConfig = fs.readFileSync('bin/plugins/lib/travis.yml', {encoding: 'utf8', flag: 'r'});
  travisConfig = travisConfig.replace(/ $p l u g i n_{n} a m e$ /g, pluginName);

  if (files.indexOf('.travis.yml') === -1) {
    console.warn('.travis.yml file not found, please create.  .travis.yml is used for automatically CI testing Etherpad.  It is useful to know if your plugin breaks another feature for example.');
    // TODO: Make it check version of the .travis file to see if it needs an update.
    if (autoFix) {
      hasAutoFixed = true;
      console.log('Autofixing missing .travis.yml file');
      fs.writeFileSync(`${pluginPath}/.travis.yml`, travisConfig);
      console.log('Travis file created, please sign into travis and enable this repository');
    }
  }
  if (autoFix && autoUpdate) {
    // checks the file versioning of .travis and updates it to the latest.
    const existingConfig = fs.readFileSync(`${pluginPath}/.travis.yml`, {encoding: 'utf8', flag: 'r'});
    const existingConfigLocation = existingConfig.indexOf('##ETHERPAD_TRAVIS_V=');
    const existingValue = parseInt(existingConfig.substr(existingConfigLocation + 20, existingConfig.length));

    const newConfigLocation = travisConfig.indexOf('##ETHERPAD_TRAVIS_V=');
    const newValue = parseInt(travisConfig.substr(newConfigLocation + 20, travisConfig.length));
    if (existingConfigLocation === -1) {
      console.warn('no previous .travis.yml version found so writing new.');
      // we will write the newTravisConfig to the location.
      fs.writeFileSync(`${pluginPath}/.travis.yml`, travisConfig);
    } else if (newValue > existingValue) {
      console.log('updating .travis.yml');
      fs.writeFileSync(`${pluginPath}/.travis.yml`, travisConfig);
      hasAutoFixed = true;
    }//
  }

  if (files.indexOf('.gitignore') === -1) {
    console.warn(".gitignore file not found, please create.  .gitignore files are useful to ensure files aren't incorrectly commited to a repository.");
    if (autoFix) {
      hasAutoFixed = true;
      console.log('Autofixing missing .gitignore file');
      const gitignore = fs.readFileSync('bin/plugins/lib/gitignore', {encoding: 'utf8', flag: 'r'});
      fs.writeFileSync(`${pluginPath}/.gitignore`, gitignore);
    }
  } else {
    let gitignore =
        fs.readFileSync(`${pluginPath}/.gitignore`, {encoding: 'utf8', flag: 'r'});
    if (gitignore.indexOf('node_modules/') === -1) {
      console.warn('node_modules/ missing from .gitignore');
      if (autoFix) {
        gitignore += 'node_modules/';
        fs.writeFileSync(`${pluginPath}/.gitignore`, gitignore);
        hasAutoFixed = true;
      }
    }
  }

  // if we include templates but don't have translations...
  if (files.indexOf('templates') !== -1 && files.indexOf('locales') === -1) {
    console.warn('Translations not found, please create.  Translation files help with Etherpad accessibility.');
  }


  if (files.indexOf('.ep_initialized') !== -1) {
    console.warn('.ep_initialized found, please remove.  .ep_initialized should never be commited to git and should only exist once the plugin has been executed one time.');
    if (autoFix) {
      hasAutoFixed = true;
      console.log('Autofixing incorrectly existing .ep_initialized file');
      fs.unlinkSync(`${pluginPath}/.ep_initialized`);
    }
  }

  if (files.indexOf('npm-debug.log') !== -1) {
    console.warn('npm-debug.log found, please remove.  npm-debug.log should never be commited to your repository.');
    if (autoFix) {
      hasAutoFixed = true;
      console.log('Autofixing incorrectly existing npm-debug.log file');
      fs.unlinkSync(`${pluginPath}/npm-debug.log`);
    }
  }

  if (files.indexOf('static') !== -1) {
    fs.readdir(`${pluginPath}/static`, (errRead, staticFiles) => {
      if (staticFiles.indexOf('tests') === -1) {
        console.warn('Test files not found, please create tests.  https://github.com/ether/etherpad-lite/wiki/Creating-a-plugin#writing-and-running-front-end-tests-for-your-plugin');
      }
    });
  } else {
    console.warn('Test files not found, please create tests.  https://github.com/ether/etherpad-lite/wiki/Creating-a-plugin#writing-and-running-front-end-tests-for-your-plugin');
  }

  // linting begins
  if (autoFix) {
    var lintCmd = 'npm run lint:fix';
  } else {
    var lintCmd = 'npm run lint';
  }

  try {
    child_process.execSync(lintCmd, {cwd: `${pluginPath}/`});
    console.log('Linting...');
    if (autoFix) {
      // todo: if npm run lint doesn't do anything no need for...
      hasAutoFixed = true;
    }
  } catch (e) {
    // it is gonna throw an error anyway
    console.log('Manual linting probably required, check with: npm run lint');
  }
  // linting ends.

  if (hasAutoFixed) {
    console.log('Fixes applied, please check git diff then run the following command:\n\n');
    // bump npm Version
    if (autoCommit) {
      // holy shit you brave.
      console.log('Attempting autocommit and auto publish to npm');
      // github should push to npm for us :)
      exec(`cd node_modules/${pluginName} && git rm -rf node_modules --ignore-unmatch && git add -A && git commit --allow-empty -m 'autofixes from Etherpad checkPlugins.js' && git push && cd ../..`, (error, name, stderr) => {
        if (error) {
          console.log(`error: ${error.message}`);
          return;
        }
        if (stderr) {
          console.log(`stderr: ${stderr}`);
          return;
        }
        console.log("I think she's got it! By George she's got it!");
        process.exit(0);
      });
    } else {
      console.log(`cd node_modules/${pluginName} && git add -A && git commit --allow-empty -m 'autofixes from Etherpad checkPlugins.js' && npm version patch && git add package.json && git commit --allow-empty -m 'bump version' && git push && npm publish && cd ../..`);
    }
  }

  console.log('Finished');
 });
--- a/bin/plugins/getCorePlugins.sh
+++ b/bin/plugins/getCorePlugins.sh
@ -0,0 +1,4 @@
 cd node_modules/
 GHUSER=ether; curl "https://api.github.com/users/$GHUSER/repos?per_page=100" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
 GHUSER=ether; curl "https://api.github.com/users/$GHUSER/repos?per_page=100&page=2&" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
 GHUSER=ether; curl "https://api.github.com/users/$GHUSER/repos?per_page=100&page=3&" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
--- a/bin/plugins/lib/CONTRIBUTING.md
+++ b/bin/plugins/lib/CONTRIBUTING.md
@ -0,0 +1,133 @@
 # Contributor Guidelines
 (Please talk to people on the mailing list before you change this page, see our section on [how to get in touch](https://github.com/ether/etherpad-lite#get-in-touch))

 ## Pull requests

 * the commit series in the PR should be _linear_ (it **should not contain merge commits**). This is necessary because we want to be able to [bisect](https://en.wikipedia.org/wiki/Bisection_(software_engineering)) bugs easily. Rewrite history/perform a rebase if necessary
 * PRs should be issued against the **develop** branch: we never pull directly into **master**
 * PRs **should not have conflicts** with develop. If there are, please resolve them rebasing and force-pushing
 * when preparing your PR, please make sure that you have included the relevant **changes to the documentation** (preferably with usage examples)
 * contain meaningful and detailed **commit messages** in the form:
  ```
  submodule: description

  longer description of the change you have made, eventually mentioning the
  number of the issue that is being fixed, in the form: Fixes #someIssueNumber
  ```
 * if the PR is a **bug fix**:
  * the first commit in the series must be a test that shows the failure
  * subsequent commits will fix the bug and make the test pass
  * the final commit message should include the text `Fixes: #xxx` to link it to its bug report
 * think about stability: code has to be backwards compatible as much as possible. Always **assume your code will be run with an older version of the DB/config file**
 * if you want to remove a feature, **deprecate it instead**:
  * write an issue with your deprecation plan
  * output a `WARN` in the log informing that the feature is going to be removed
  * remove the feature in the next version
 * if you want to add a new feature, put it under a **feature flag**:
  * once the new feature has reached a minimal level of stability, do a PR for it, so it can be integrated early
  * expose a mechanism for enabling/disabling the feature
  * the new feature should be **disabled** by default. With the feature disabled, the code path should be exactly the same as before your contribution. This is a __necessary condition__ for early integration
 * think of the PR not as something that __you wrote__, but as something that __someone else is going to read__. The commit series in the PR should tell a novice developer the story of your thoughts when developing it

 ## How to write a bug report

 * Please be polite, we all are humans and problems can occur.
 * Please add as much information as possible, for example
  * client os(s) and version(s)
    * browser(s) and version(s), is the problem reproducible on different clients
    * special environments like firewalls or antivirus
  * host os and version
    * npm and nodejs version
    * Logfiles if available
  * steps to reproduce
  * what you expected to happen
  * what actually happened
 * Please format logfiles and code examples with markdown see github Markdown help below the issue textarea for more information.

 If you send logfiles, please set the loglevel switch DEBUG in your settings.json file:

 ```
 /* The log level we are using, can be: DEBUG, INFO, WARN, ERROR */
  "loglevel": "DEBUG",
 ```

 The logfile location is defined in startup script or the log is directly shown in the commandline after you have started etherpad.

 ## General goals of Etherpad
 To make sure everybody is going in the same direction:
 * easy to install for admins and easy to use for people
 * easy to integrate into other apps, but also usable as standalone
 * lightweight and scalable
 * extensible, as much functionality should be extendable with plugins so changes don't have to be done in core.
 Also, keep it maintainable. We don't wanna end up as the monster Etherpad was!

 ## How to work with git?
 * Don't work in your master branch.
 * Make a new branch for every feature you're working on. (This ensures that you can work you can do lots of small, independent pull requests instead of one big one with complete different features)
 * Don't use the online edit function of github (this only creates ugly and not working commits!)
 * Try to make clean commits that are easy readable (including descriptive commit messages!)
 * Test before you push. Sounds easy, it isn't!
 * Don't check in stuff that gets generated during build or runtime
 * Make small pull requests that are easy to review but make sure they do add value by themselves / individually

 ## Coding style
 * Do write comments. (You don't have to comment every line, but if you come up with something that's a bit complex/weird, just leave a comment. Bear in mind that you will probably leave the project at some point and that other people will read your code. Undocumented huge amounts of code are worthless!)
 * Never ever use tabs
 * Indentation: JS/CSS: 2 spaces; HTML: 4 spaces
 * Don't overengineer. Don't try to solve any possible problem in one step, but try to solve problems as easy as possible and improve the solution over time!
 * Do generalize sooner or later! (if an old solution, quickly hacked together, poses more problems than it solves today, refactor it!)
 * Keep it compatible. Do not introduce changes to the public API, db schema or configurations too lightly. Don't make incompatible changes without good reasons!
 * If you do make changes, document them! (see below)
 * Use protocol independent urls "//"

 ## Branching model / git workflow
 see git flow http://nvie.com/posts/a-successful-git-branching-model/

 ### `master` branch
 * the stable
 * This is the branch everyone should use for production stuff

 ### `develop`branch
 * everything that is READY to go into master at some point in time
 * This stuff is tested and ready to go out

 ### release branches
 * stuff that should go into master very soon
 * only bugfixes go into these (see http://nvie.com/posts/a-successful-git-branching-model/ for why)
 * we should not be blocking new features to develop, just because we feel that we should be releasing it to master soon. This is the situation that release branches solve/handle.

 ### hotfix branches
 * fixes for bugs in master

 ### feature branches (in your own repos)
 * these are the branches where you develop your features in
 * If it's ready to go out, it will be merged into develop

 Over the time we pull features from feature branches into the develop branch. Every month we pull from develop into master. Bugs in master get fixed in hotfix branches. These branches will get merged into master AND develop. There should never be commits in master that aren't in develop

 ## Documentation
 The docs are in the `doc/` folder in the git repository, so people can easily find the suitable docs for the current git revision.

 Documentation should be kept up-to-date. This means, whenever you add a new API method, add a new hook or change the database model, pack the relevant changes to the docs in the same pull request.

 You can build the docs e.g. produce html, using `make docs`. At some point in the future we will provide an online documentation. The current documentation in the github wiki should always reflect the state of `master` (!), since there are no docs in master, yet.

 ## Testing
 Front-end tests are found in the `tests/frontend/` folder in the repository. Run them by pointing your browser to `<yourdomainhere>/tests/frontend`.

 Back-end tests can be run from the `src` directory, via `npm test`.

 ## Things you can help with
 Etherpad is much more than software.  So if you aren't a developer then worry not, there is still a LOT you can do!  A big part of what we do is community engagement.  You can help in the following ways
 * Triage bugs (applying labels) and confirming their existence
 * Testing fixes (simply applying them and seeing if it fixes your issue or not) - Some git experience required
 * Notifying large site admins of new releases
 * Writing Changelogs for releases
 * Creating Windows packages
 * Creating releases
 * Bumping dependencies periodically and checking they don't break anything
 * Write proposals for grants
 * Co-Author and Publish CVEs
 * Work with SFC to maintain legal side of project
 * Maintain TODO page - https://github.com/ether/etherpad-lite/wiki/TODO#IMPORTANT_TODOS

--- a/bin/plugins/lib/LICENSE.md
+++ b/bin/plugins/lib/LICENSE.md
@ -0,0 +1,13 @@
 Copyright [yyyy] [name of copyright owner]

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
--- a/bin/plugins/lib/README.md
+++ b/bin/plugins/lib/README.md
@ -0,0 +1,29 @@
 [![Travis (.com)](https://api.travis-ci.com/[org_name]/[repo_url].svg?branch=develop)](https://travis-ci.com/github/[org_name]/[repo_url])

 # My awesome plugin README example
 Explain what your plugin does and who it's useful for.

 ## Example animated gif of usage if appropriate
 ![screenshot](https://user-images.githubusercontent.com/220864/99979953-97841d80-2d9f-11eb-9782-5f65817c58f4.PNG)

 ## Installing
 npm install [plugin_name]

 or Use the Etherpad ``/admin`` interface.

 ## Settings
 Document settings if any

 ## Testing
 Document how to run backend / frontend tests.

 ### Frontend

 Visit http://whatever/tests/frontend/ to run the frontend tests.

 ### backend

 Type ``cd src && npm run test`` to run the backend tests.

 ## LICENSE
 Apache 2.0
--- a/bin/plugins/lib/backend-tests.yml
+++ b/bin/plugins/lib/backend-tests.yml
@ -0,0 +1,51 @@
 # You need to change lines 38 and 46 in case the plugin's name on npmjs.com is different
 # from the repository name

 name: "Backend tests"

 # any branch is useful for testing before a PR is submitted
 on: [push, pull_request]

 jobs:
  withplugins:
    # run on pushes to any branch
    # run on PRs from external forks
    if: |
         (github.event_name != 'pull_request')
         || (github.event.pull_request.head.repo.id != github.event.pull_request.base.repo.id)
    name: with Plugins
    runs-on: ubuntu-latest

    steps:
    - name: Install libreoffice
      run: |
        sudo add-apt-repository -y ppa:libreoffice/ppa
        sudo apt update
        sudo apt install -y --no-install-recommends libreoffice libreoffice-pdfimport

    # clone etherpad-lite
    - name: Install etherpad core
      uses: actions/checkout@v2
      with:
        repository: ether/etherpad-lite

    - name: Install all dependencies and symlink for ep_etherpad-lite
      run: bin/installDeps.sh

    # clone this repository into node_modules/ep_plugin-name
    - name: Checkout plugin repository
      uses: actions/checkout@v2
      with:
        path: ./node_modules/${{github.event.repository.name}}

    - name: Install plugin dependencies
      run: |
        cd node_modules/${{github.event.repository.name}}
        npm ci

    # configures some settings and runs npm run test
    - name: Run the backend tests
      run: tests/frontend/travis/runnerBackend.sh

 ##ETHERPAD_NPM_V=1
 ## NPM configuration automatically created using bin/plugins/updateAllPluginsScript.sh
--- a/bin/plugins/lib/gitignore
+++ b/bin/plugins/lib/gitignore
@ -0,0 +1,5 @@
 .ep_initialized
 .DS_Store
 node_modules/
 node_modules
 npm-debug.log
--- a/bin/plugins/lib/npmpublish.yml
+++ b/bin/plugins/lib/npmpublish.yml
@ -0,0 +1,73 @@
 # This workflow will run tests using node and then publish a package to the npm registry when a release is created
 # For more information see: https://help.github.com/actions/language-and-framework-guides/publishing-nodejs-packages

 name: Node.js Package

 on:
  pull_request:
  push:
    branches:
      - main
      - master

 jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      # Clone ether/etherpad-lite to ../etherpad-lite so that ep_etherpad-lite
      # can be "installed" in this plugin's node_modules. The checkout v2 action
      # doesn't support cloning outside of $GITHUB_WORKSPACE (see
      # https://github.com/actions/checkout/issues/197), so the repo is first
      # cloned to etherpad-lite then moved to ../etherpad-lite. To avoid
      # conflicts with this plugin's clone, etherpad-lite must be cloned and
      # moved out before this plugin's repo is cloned to $GITHUB_WORKSPACE.
      - uses: actions/checkout@v2
        with:
          repository: ether/etherpad-lite
          path: etherpad-lite
      - run: mv etherpad-lite ..
      # etherpad-lite has been moved outside of $GITHUB_WORKSPACE, so it is now
      # safe to clone this plugin's repo to $GITHUB_WORKSPACE.
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 12
      # All of ep_etherpad-lite's devDependencies are installed because the
      # plugin might do `require('ep_etherpad-lite/node_modules/${devDep}')`.
      # Eventually it would be nice to create an ESLint plugin that prohibits
      # Etherpad plugins from piggybacking off of ep_etherpad-lite's
      # devDependencies. If we had that, we could change this line to only
      # install production dependencies.
      - run: cd ../etherpad-lite/src && npm ci
      - run: npm ci
      # This runs some sanity checks and creates a symlink at
      # node_modules/ep_etherpad-lite that points to ../../etherpad-lite/src.
      # This step must be done after `npm ci` installs the plugin's dependencies
      # because npm "helpfully" cleans up such symlinks. :( Installing
      # ep_etherpad-lite in the plugin's node_modules prevents lint errors and
      # unit test failures if the plugin does `require('ep_etherpad-lite/foo')`.
      - run: npm install --no-save ep_etherpad-lite@file:../etherpad-lite/src
      - run: npm test
      - run: npm run lint

  publish-npm:
    if: github.event_name == 'push'
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 12
          registry-url: https://registry.npmjs.org/
      - run: git config user.name 'github-actions[bot]'
      - run: git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
      - run: npm ci
      - run: npm version patch
      - run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
      - run: git push --follow-tags

 ##ETHERPAD_NPM_V=1
 ## NPM configuration automatically created using bin/plugins/updateAllPluginsScript.sh
--- a/bin/plugins/lib/travis.yml
+++ b/bin/plugins/lib/travis.yml
@ -0,0 +1,70 @@
 language: node_js

 node_js:
  - "lts/*"

 cache: false

 services:
  - docker

 install:
  - "export GIT_HASH=$(git rev-parse --verify --short HEAD)"

 #script:
 #  - "tests/frontend/travis/runner.sh"

 env:
  global:
    - secure: "WMGxFkOeTTlhWB+ChMucRtIqVmMbwzYdNHuHQjKCcj8HBEPdZLfCuK/kf4rG\nVLcLQiIsyllqzNhBGVHG1nyqWr0/LTm8JRqSCDDVIhpyzp9KpCJQQJG2Uwjk\n6/HIJJh/wbxsEdLNV2crYU/EiVO3A4Bq0YTHUlbhUqG3mSCr5Ec="
    - secure: "gejXUAHYscbR6Bodw35XexpToqWkv2ifeECsbeEmjaLkYzXmUUNWJGknKSu7\nEUsSfQV8w+hxApr1Z+jNqk9aX3K1I4btL3cwk2trnNI8XRAvu1c1Iv60eerI\nkE82Rsd5lwUaMEh+/HoL8ztFCZamVndoNgX7HWp5J/NRZZMmh4g="

 jobs:
  include:
    - name: "Lint test package-lock"
      install:
        - "npm install lockfile-lint"
      script:
        - npx lockfile-lint --path package-lock.json --validate-https --allowed-hosts npm
    - name: "Run the Backend tests"
      before_install:
        - sudo add-apt-repository -y ppa:libreoffice/ppa
        - sudo apt-get update
        - sudo apt-get -y install libreoffice
        - sudo apt-get -y install libreoffice-pdfimport
      install:
        - "npm install"
        - "mkdir [plugin_name]"
        - "mv !([plugin_name]) [plugin_name]"
        - "git clone https://github.com/ether/etherpad-lite.git etherpad"
        - "cd etherpad"
        - "mkdir -p node_modules"
        - "mv ../[plugin_name] node_modules"
        - "bin/installDeps.sh"
        - "export GIT_HASH=$(git rev-parse --verify --short HEAD)"
        - "cd src && npm install && cd -"
      script:
        - "tests/frontend/travis/runnerBackend.sh"
    - name: "Test the Frontend"
      before_script:
        - "tests/frontend/travis/sauce_tunnel.sh"
      install:
        - "npm install"
        - "mkdir [plugin_name]"
        - "mv !([plugin_name]) [plugin_name]"
        - "git clone https://github.com/ether/etherpad-lite.git etherpad"
        - "cd etherpad"
        - "mkdir -p node_modules"
        - "mv ../[plugin_name] node_modules"
        - "bin/installDeps.sh"
        - "export GIT_HASH=$(git rev-parse --verify --short HEAD)"
      script:
        - "tests/frontend/travis/runner.sh"

 notifications:
  irc:
    channels:
      - "irc.freenode.org#etherpad-lite-dev"

 ##ETHERPAD_TRAVIS_V=9
 ## Travis configuration automatically created using bin/plugins/updateAllPluginsScript.sh
--- a/bin/plugins/reTestAllPlugins.sh
+++ b/bin/plugins/reTestAllPlugins.sh
@ -0,0 +1,14 @@
 echo "herp";
 for dir in `ls node_modules`;
 do
  echo $dir
  if [[ $dir == *"ep_"* ]]; then
    if [[ $dir != "ep_etherpad-lite" ]]; then
      # node bin/plugins/checkPlugin.js $dir autofix autocommit autoupdate
      cd node_modules/$dir
      git commit -m "Automatic update: bump update to re-run latest Etherpad tests" --allow-empty
      git push origin master
      cd ../..
    fi
  fi
 done
--- a/bin/plugins/updateAllPluginsScript.sh
+++ b/bin/plugins/updateAllPluginsScript.sh
@ -0,0 +1,17 @@
 cd node_modules
 GHUSER=johnmclear; curl "https://api.github.com/users/$GHUSER/repos?per_page=1000" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
 GHUSER=johnmclear; curl "https://api.github.com/users/$GHUSER/repos?per_page=1000&page=2" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
 GHUSER=johnmclear; curl "https://api.github.com/users/$GHUSER/repos?per_page=1000&page=3" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
 GHUSER=johnmclear; curl "https://api.github.com/users/$GHUSER/repos?per_page=1000&page=4" | grep -o 'git@[^"]*' | grep /ep_ | xargs -L1 git clone
 cd ..

 for dir in `ls node_modules`;
 do
  # echo $0
  if [[ $dir == *"ep_"* ]]; then
    if [[ $dir != "ep_etherpad-lite" ]]; then
      node bin/plugins/checkPlugin.js $dir autofix autocommit autoupdate
    fi
  fi
  # echo $dir
 done
--- a/bin/plugins/updateCorePlugins.sh
+++ b/bin/plugins/updateCorePlugins.sh
@ -0,0 +1,9 @@
 #!/bin/sh

 set -e

 for dir in node_modules/ep_*; do
  dir=${dir#node_modules/}
  [ "$dir" != ep_etherpad-lite ] || continue
  node bin/plugins/checkPlugin.js "$dir" autofix autocommit autoupdate
 done
--- a/bin/rebuildPad.js
+++ b/bin/rebuildPad.js
@ -0,0 +1,126 @@
 /*
  This is a repair tool. It rebuilds an old pad at a new pad location up to a
  known "good" revision.
 */

 if (process.argv.length != 4 && process.argv.length != 5) {
  console.error('Use: node bin/repairPad.js  $P A D I D$ REV [$NEWPADID]');
  process.exit(1);
 }

 const npm = require('../src/node_modules/npm');
 const async = require('../src/node_modules/async');
 const ueberDB = require('../src/node_modules/ueberdb2');

 const padId = process.argv[2];
 const newRevHead = process.argv[3];
 const newPadId = process.argv[4] || `${padId}-rebuilt`;

 let db, oldPad, newPad, settings;
 let AuthorManager, ChangeSet, Pad, PadManager;

 async.series([
  function (callback) {
    npm.load({}, (err) => {
      if (err) {
        console.error(`Could not load NPM: ${err}`);
        process.exit(1);
      } else {
        callback();
      }
    });
  },
  function (callback) {
    // Get a handle into the database
    db = require('../src/node/db/DB');
    db.init(callback);
  },
  function (callback) {
    PadManager = require('../src/node/db/PadManager');
    Pad = require('../src/node/db/Pad').Pad;
    // Get references to the original pad and to a newly created pad
    // HACK: This is a standalone script, so we want to write everything
    // out to the database immediately.  The only problem with this is
    // that a driver (like the mysql driver) can hardcode these values.
    db.db.db.settings = {cache: 0, writeInterval: 0, json: true};
    // Validate the newPadId if specified and that a pad with that ID does
    // not already exist to avoid overwriting it.
    if (!PadManager.isValidPadId(newPadId)) {
      console.error('Cannot create a pad with that id as it is invalid');
      process.exit(1);
    }
    PadManager.doesPadExists(newPadId, (err, exists) => {
      if (exists) {
        console.error('Cannot create a pad with that id as it already exists');
        process.exit(1);
      }
    });
    PadManager.getPad(padId, (err, pad) => {
      oldPad = pad;
      newPad = new Pad(newPadId);
      callback();
    });
  },
  function (callback) {
    // Clone all Chat revisions
    const chatHead = oldPad.chatHead;
    for (var i = 0, curHeadNum = 0; i <= chatHead; i++) {
      db.db.get(`pad:${padId}:chat:${i}`, (err, chat) => {
        db.db.set(`pad:${newPadId}:chat:${curHeadNum++}`, chat);
        console.log(`Created: Chat Revision: pad:${newPadId}:chat:${curHeadNum}`);
      });
    }
    callback();
  },
  function (callback) {
    // Rebuild Pad from revisions up to and including the new revision head
    AuthorManager = require('../src/node/db/AuthorManager');
    Changeset = require('ep_etherpad-lite/static/js/Changeset');
    // Author attributes are derived from changesets, but there can also be
    // non-author attributes with specific mappings that changesets depend on
    // and, AFAICT, cannot be recreated any other way
    newPad.pool.numToAttrib = oldPad.pool.numToAttrib;
    for (let curRevNum = 0; curRevNum <= newRevHead; curRevNum++) {
      db.db.get(`pad:${padId}:revs:${curRevNum}`, (err, rev) => {
        if (rev.meta) {
          throw 'The specified revision number could not be found.';
        }
        const newRevNum = ++newPad.head;
        const newRevId = `pad:${newPad.id}:revs:${newRevNum}`;
        db.db.set(newRevId, rev);
        AuthorManager.addPad(rev.meta.author, newPad.id);
        newPad.atext = Changeset.applyToAText(rev.changeset, newPad.atext, newPad.pool);
        console.log(`Created: Revision: pad:${newPad.id}:revs:${newRevNum}`);
        if (newRevNum == newRevHead) {
          callback();
        }
      });
    }
  },
  function (callback) {
    // Add saved revisions up to the new revision head
    console.log(newPad.head);
    const newSavedRevisions = [];
    for (const i in oldPad.savedRevisions) {
      savedRev = oldPad.savedRevisions[i];
      if (savedRev.revNum <= newRevHead) {
        newSavedRevisions.push(savedRev);
        console.log(`Added: Saved Revision: ${savedRev.revNum}`);
      }
    }
    newPad.savedRevisions = newSavedRevisions;
    callback();
  },
  function (callback) {
    // Save the source pad
    db.db.set(`pad:${newPadId}`, newPad, (err) => {
      console.log(`Created: Source Pad: pad:${newPadId}`);
      newPad.saveToDatabase().then(() => callback(), callback);
    });
  },
 ], (err) => {
  if (err) { throw err; } else {
    console.info('finished');
    process.exit(0);
  }
 });
--- a/bin/release.js
+++ b/bin/release.js
@ -0,0 +1,67 @@
 'use strict';

 const fs = require('fs');
 const child_process = require('child_process');
 const semver = require('../src/node_modules/semver');

 /*

 Usage

 node bin/release.js patch

 */
 const usage = 'node bin/release.js [patch/minor/major] -- example: "node bin/release.js patch"';

 const release = process.argv[2];

 if(!release) {
  console.log(usage);
  throw new Error('No release type included');
 }

 const changelog = fs.readFileSync('CHANGELOG.md', {encoding: 'utf8', flag: 'r'});
 let packageJson = fs.readFileSync('./src/package.json', {encoding: 'utf8', flag: 'r'});
 packageJson = JSON.parse(packageJson);
 const currentVersion = packageJson.version;

 const newVersion = semver.inc(currentVersion, release);
 if(!newVersion) {
  console.log(usage);
  throw new Error('Unable to generate new version from input');
 }

 const changelogIncludesVersion = changelog.indexOf(newVersion) !== -1;

 if(!changelogIncludesVersion) {
  throw new Error('No changelog record for ', newVersion, ' - please create changelog record');
 }

 console.log('Okay looks good, lets create the package.json and package-lock.json');

 packageJson.version = newVersion;

 fs.writeFileSync('src/package.json', JSON.stringify(packageJson, null, 2));

 // run npm version `release` where release is patch, minor or major
 child_process.execSync('npm install --package-lock-only', {cwd: `src/`});
 // run npm install --package-lock-only <-- required???

 child_process.execSync(`git checkout -b release/${newVersion}`);
 child_process.execSync(`git add src/package.json`);
 child_process.execSync(`git add src/package-lock.json`);
 child_process.execSync(`git commit -m 'bump version'`);
 child_process.execSync(`git push origin release/${newVersion}`);


 child_process.execSync(`make docs`);
 child_process.execSync(`git clone git@github.com:ether/ether.github.com.git`);
 child_process.execSync(`cp -R out/doc/ ether.github.com/doc/${newVersion}`);

 console.log('Once merged into master please run the following commands');
 console.log(`git tag -a ${newVersion} -m ${newVersion} && git push origin master`);
 console.log(`cd ether.github.com && git add . && git commit -m '${newVersion} docs'`);
 console.log(`Build the windows zip`)
 console.log(`Visit https://github.com/ether/etherpad-lite/releases/new and create a new release with 'master' as the target and the version is ${newVersion}.  Include the windows zip as an assett`)
 console.log('Once the new docs are uploaded then modify the download link on etherpad.org and then pull master onto develop');
 console.log('Finally go public with an announcement via our comms channels :)');
--- a/bin/repairPad.js
+++ b/bin/repairPad.js
@ -0,0 +1,77 @@
 /*
 * This is a repair tool. It extracts all datas of a pad, removes and inserts them again.
 */

 console.warn('WARNING: This script must not be used while etherpad is running!');

 if (process.argv.length != 3) {
  console.error('Use: node bin/repairPad.js $PADID');
  process.exit(1);
 }

 // get the padID
 const padId = process.argv[2];

 const npm = require('../src/node_modules/npm');
 npm.load({}, async (er) => {
  if (er) {
    console.error(`Could not load NPM: ${er}`);
    process.exit(1);
  }

  try {
    // intialize database
    const settings = require('../src/node/utils/Settings');
    const db = require('../src/node/db/DB');
    await db.init();

    // get the pad
    const padManager = require('../src/node/db/PadManager');
    const pad = await padManager.getPad(padId);

    // accumulate the required keys
    const neededDBValues = [`pad:${padId}`];

    // add all authors
    neededDBValues.push(...pad.getAllAuthors().map((author) => 'globalAuthor:'));

    // add all revisions
    for (let rev = 0; rev <= pad.head; ++rev) {
      neededDBValues.push(`pad:${padId}:revs:${rev}`);
    }

    // add all chat values
    for (let chat = 0; chat <= pad.chatHead; ++chat) {
      neededDBValues.push(`pad:${padId}:chat:${chat}`);
    }

    //
    // NB: this script doesn't actually does what's documented
    //     since the `value` fields in the following `.forEach`
    //     block are just the array index numbers
    //
    //     the script therefore craps out now before it can do
    //     any damage.
    //
    //     See gitlab issue #3545
    //
    console.info('aborting [gitlab #3545]');
    process.exit(1);

    // now fetch and reinsert every key
    neededDBValues.forEach((key, value) => {
      console.log(`Key: ${key}, value: ${value}`);
      db.remove(key);
      db.set(key, value);
    });

    console.info('finished');
    process.exit(0);
  } catch (er) {
    if (er.name === 'apierror') {
      console.error(er);
    } else {
      console.trace(er);
    }
  }
 });
--- a/bin/run.sh
+++ b/bin/run.sh
@ -0,0 +1,35 @@
 #!/bin/sh

 # Move to the folder where ep-lite is installed
 cd "$(dirname "$0")"/..

 # Source constants and usefull functions
 . bin/functions.sh

 ignoreRoot=0
 for ARG in "$@"; do
  if [ "$ARG" = "--root" ]; then
    ignoreRoot=1
  fi
 done

 # Stop the script if it's started as root
 if [ "$(id -u)" -eq 0 ] && [ "$ignoreRoot" -eq 0 ]; then
  cat <<EOF >&2
 You shouldn't start Etherpad as root!
 Please type 'Etherpad rocks my socks' (or restart with the '--root'
 argument) if you still want to start it as root:
 EOF
  printf "> " >&2
  read rocks
  [ "$rocks" = "Etherpad rocks my socks" ] || fatal "Your input was incorrect"
 fi

 # Prepare the environment
 bin/installDeps.sh "$@" || exit 1

 # Move to the node folder and start
 log "Starting Etherpad..."

 SCRIPTPATH=$(pwd -P)
 exec node $(compute_node_args) "$SCRIPTPATH/node_modules/ep_etherpad-lite/node/server.js" "$@"
--- a/bin/safeRun.sh
+++ b/bin/safeRun.sh
@ -0,0 +1,69 @@
 #!/bin/sh

 # This script ensures that ep-lite is automatically restarting after
 # an error happens

 # Handling Errors
 #   0 silent
 #   1 email
 ERROR_HANDLING=0
 # Your email address which should receive the error messages
 EMAIL_ADDRESS="no-reply@example.com"
 # Sets the minimum amount of time between the sending of error emails.
 # This ensures you do not get spammed during an endless reboot loop
 # It's the time in seconds
 TIME_BETWEEN_EMAILS=600 # 10 minutes

 # DON'T EDIT AFTER THIS LINE

 pecho() { printf %s\n "$*"; }
 log() { pecho "$@"; }
 error() { log "ERROR: $@" >&2; }
 fatal() { error "$@"; exit 1; }

 LAST_EMAIL_SEND=0

 # Move to the folder where ep-lite is installed
 cd "$(dirname "$0")"/..

 # Check if a logfile parameter is set
 LOG="$1"
 [ -n "${LOG}" ] || fatal "Set a logfile as the first parameter"
 shift

 while true; do
  # Try to touch the file if it doesn't exist
  [ -f "${LOG}" ] || touch "${LOG}" || fatal "Logfile '${LOG}' is not writeable"

  # Check if the file is writeable
  [ -w "${LOG}" ] || fatal "Logfile '${LOG}' is not writeable"

  # Start the application
  bin/run.sh "$@" >>${LOG} 2>>${LOG}

  TIME_FMT=$(date +%Y-%m-%dT%H:%M:%S%z)

  # Send email
  if [ "$ERROR_HANDLING" = 1 ]; then
    TIME_NOW=$(date +%s)
    TIME_SINCE_LAST_SEND=$(($TIME_NOW - $LAST_EMAIL_SEND))

    if [ "$TIME_SINCE_LAST_SEND" -gt "$TIME_BETWEEN_EMAILS" ]; then
      {
        cat <<EOF
 Server was restarted at: ${TIME_FMT}
 The last 50 lines of the log before the server exited:

 EOF
        tail -n 50 "${LOG}"
      } | mail -s "Etherpad restarted" "$EMAIL_ADDRESS"

      LAST_EMAIL_SEND=$TIME_NOW
    fi
  fi

  pecho "RESTART! ${TIME_FMT}" >>${LOG}

  # Sleep 10 seconds before restart
  sleep 10
 done
--- a/bin/updatePlugins.sh
+++ b/bin/updatePlugins.sh
@ -0,0 +1,20 @@
 #!/bin/sh

 #Move to the folder where ep-lite is installed
 cd $(dirname $0)

 #Was this script started in the bin folder? if yes move out
 if [ -d "../bin" ]; then
  cd "../"
 fi

 # npm outdated --depth=0 | grep -v "^Package" | awk '{print  $Extra close brace or missing open brace$ 1 --save-dev
 OUTDATED=$(npm outdated --depth=0 | grep -v "^Package" | awk '{print $1}')
 # echo $OUTDATED
 if test -n "$OUTDATED"; then
  echo "Plugins require update, doing this now..."
  echo "Updating $OUTDATED"
  npm install $OUTDATED --save-dev
 else
  echo "Plugins are all up to date"
 fi
--- a/doc/api/api.md
+++ b/doc/api/api.md
@ -0,0 +1,10 @@
@include embed_parameters
@include http_api
@include hooks_overview
@include hooks_client-side
@include hooks_server-side
@include editorInfo
@include changeset_library
@include pluginfw
@include toolbar
@include editbar
--- a/doc/api/changeset_library.md
+++ b/doc/api/changeset_library.md
@ -0,0 +1,151 @@
 # Changeset Library

 ```
 "Z:z>1|2=m=b*0|1+1$\n"
 ```

 This is a Changeset. It's just a string and it's very difficult to read in this form. But the Changeset Library gives us some tools to read it.

 A changeset describes the diff between two revisions of the document. The Browser sends changesets to the server and the server sends them to the clients to update them. These Changesets also get saved into the history of a pad. This allows us to go back to every revision from the past.

 ## Changeset.unpack(changeset)

 * `changeset` {String}

 This function returns an object representation of the changeset, similar to this:

 ```
 { oldLen: 35, newLen: 36, ops: '|2=m=b*0|1+1', charBank: '\n' }
 ```

 * `oldLen` {Number} the original length of the document.
 * `newLen` {Number} the length of the document after the changeset is applied.
 * `ops` {String} the actual changes, introduced by this changeset.
 * `charBank` {String} All characters that are added by this changeset.

 ## Changeset.opIterator(ops)

 * `ops` {String} The operators, returned by `Changeset.unpack()`

 Returns an operator iterator. This iterator allows us to iterate over all operators that are in the changeset.

 You can iterate with an opIterator using its `next()` and `hasNext()` methods. Next returns the `next()` operator object and `hasNext()` indicates, whether there are any operators left.

 ## The Operator object
 There are 3 types of operators: `+`,`-` and `=`. These operators describe different changes to the document, beginning with the first character of the document. A `=` operator doesn't change the text, but it may add or remove text attributes. A `-` operator removes text. And a `+` Operator adds text and optionally adds some attributes to it.

 * `opcode` {String} the operator type
 * `chars` {Number} the length of the text changed by this operator.
 * `lines` {Number} the number of lines changed by this operator.
 * `attribs` {attribs} attributes set on this text.

 ### Example
 ```
 { opcode: '+',
  chars: 1,
  lines: 1,
  attribs: '*0' }
 ```

 ## APool

 ```
 > var AttributePoolFactory = require("./utils/AttributePoolFactory");
 > var apool = AttributePoolFactory.createAttributePool();
 > console.log(apool)
 { numToAttrib: {},
  attribToNum: {},
  nextNum: 0,
  putAttrib: [Function],
  getAttrib: [Function],
  getAttribKey: [Function],
  getAttribValue: [Function],
  eachAttrib: [Function],
  toJsonable: [Function],
  fromJsonable: [Function] }
 ```

 This creates an empty apool. An apool saves which attributes were used during the history of a pad. There is one apool for each pad. It only saves the attributes that were really used, it doesn't save unused attributes. Let's fill this apool with some values

 ```
 > apool.fromJsonable({"numToAttrib":{"0":["author","a.kVnWeomPADAT2pn9"],"1":["bold","true"],"2":["italic","true"]},"nextNum":3});
 > console.log(apool)
 { numToAttrib:
   { '0': [ 'author', 'a.kVnWeomPADAT2pn9' ],
     '1': [ 'bold', 'true' ],
     '2': [ 'italic', 'true' ] },
  attribToNum:
   { 'author,a.kVnWeomPADAT2pn9': 0,
     'bold,true': 1,
     'italic,true': 2 },
  nextNum: 3,
  putAttrib: [Function],
  getAttrib: [Function],
  getAttribKey: [Function],
  getAttribValue: [Function],
  eachAttrib: [Function],
  toJsonable: [Function],
  fromJsonable: [Function] }
 ```

 We used the fromJsonable function to fill the empty apool with values. the fromJsonable and toJsonable functions are used to serialize and deserialize an apool. You can see that it stores the relation between numbers and attributes. So for example the attribute 1 is the attribute bold and vise versa. An attribute is always a key value pair. For stuff like bold and italic it's just  'italic':'true'. For authors it's author:$AUTHORID. So a character can be bold and italic. But it can't belong to multiple authors

 ```
 > apool.getAttrib(1)
 [ 'bold', 'true' ]
 ```

 Simple example of how to get the key value pair for the attribute 1

 ## AText

 ```
 > var atext = {"text":"bold text\nitalic text\nnormal text\n\n","attribs":"*0*1+9*0|1+1*0*1*2+b|1+1*0+b|2+2"};
 > console.log(atext)
 { text: 'bold text\nitalic text\nnormal text\n\n',
  attribs: '*0*1+9*0|1+1*0*1*2+b|1+1*0+b|2+2' }
 ```

 This is an atext. An atext has two parts: text and attribs. The text is just the text of the pad as a string. We will look closer at the attribs at the next steps

 ```
 > var opiterator = Changeset.opIterator(atext.attribs)
 > console.log(opiterator)
 { next: [Function: next],
  hasNext: [Function: hasNext],
  lastIndex: [Function: lastIndex] }
 > opiterator.next()
 { opcode: '+',
  chars: 9,
  lines: 0,
  attribs: '*0*1' }
 > opiterator.next()
 { opcode: '+',
  chars: 1,
  lines: 1,
  attribs: '*0' }
 > opiterator.next()
 { opcode: '+',
  chars: 11,
  lines: 0,
  attribs: '*0*1*2' }
 > opiterator.next()
 { opcode: '+',
  chars: 1,
  lines: 1,
  attribs: '' }
 > opiterator.next()
 { opcode: '+',
  chars: 11,
  lines: 0,
  attribs: '*0' }
 > opiterator.next()
 { opcode: '+',
  chars: 2,
  lines: 2,
  attribs: '' }
 ```

 The attribs are again a bunch of operators like .ops in the changeset was. But these operators are only + operators. They describe which part of the text has which attributes

 For more information see /doc/easysync/easysync-notes.txt in the source.
--- a/doc/api/editbar.md
+++ b/doc/api/editbar.md
@ -0,0 +1,28 @@
 # Editbar
 src/static/js/pad_editbar.js

 ## isEnabled()

 ## disable()

 ## toggleDropDown(dropdown, callback)
 Shows the dropdown `div.popup` whose `id` equals `dropdown`.

 ## registerCommand(cmd, callback)
 Register a handler for a specific command. Commands are fired if the corresponding button is clicked or the corresponding select is changed.

 ## registerAceCommand(cmd, callback)
 Creates an ace callstack and calls the callback with an ace instance (and a toolbar item, if applicable): `callback(cmd, ace, item)`.

 Example:
 ```
 toolbar.registerAceCommand("insertorderedlist", function (cmd, ace) {
  ace.ace_doInsertOrderedList();
 });
 ```

 ## registerDropdownCommand(cmd, dropdown)
 Ties a `div.popup` where `id` equals `dropdown` to a `command` fired by clicking a button.

 ## triggerCommand(cmd[, item])
 Triggers a command (optionally with some internal representation of the toolbar item that triggered it).
--- a/doc/api/editorInfo.md
+++ b/doc/api/editorInfo.md
@ -0,0 +1,83 @@
 # editorInfo

 ## editorInfo.ace_replaceRange(start, end, text)
 This function replaces a range (from `start` to `end`) with `text`.

 ## editorInfo.ace_getRep()
 Returns the `rep` object.

 ## editorInfo.ace_getAuthor()
 ## editorInfo.ace_inCallStack()
 ## editorInfo.ace_inCallStackIfNecessary(?)
 ## editorInfo.ace_focus(?)
 ## editorInfo.ace_importText(?)
 ## editorInfo.ace_importAText(?)
 ## editorInfo.ace_exportText(?)
 ## editorInfo.ace_editorChangedSize(?)
 ## editorInfo.ace_setOnKeyPress(?)
 ## editorInfo.ace_setOnKeyDown(?)
 ## editorInfo.ace_setNotifyDirty(?)
 ## editorInfo.ace_dispose(?)
 ## editorInfo.ace_getFormattedCode(?)
 ## editorInfo.ace_setEditable(bool)
 ## editorInfo.ace_execCommand(?)
 ## editorInfo.ace_callWithAce(fn, callStack, normalize)
 ## editorInfo.ace_setProperty(key, value)
 ## editorInfo.ace_setBaseText(txt)
 ## editorInfo.ace_setBaseAttributedText(atxt, apoolJsonObj)
 ## editorInfo.ace_applyChangesToBase(c, optAuthor, apoolJsonObj)
 ## editorInfo.ace_prepareUserChangeset()
 ## editorInfo.ace_applyPreparedChangesetToBase()
 ## editorInfo.ace_setUserChangeNotificationCallback(f)
 ## editorInfo.ace_setAuthorInfo(author, info)
 ## editorInfo.ace_setAuthorSelectionRange(author, start, end)
 ## editorInfo.ace_getUnhandledErrors()
 ## editorInfo.ace_getDebugProperty(prop)
 ## editorInfo.ace_fastIncorp(?)
 ## editorInfo.ace_isCaret(?)
 ## editorInfo.ace_getLineAndCharForPoint(?)
 ## editorInfo.ace_performDocumentApplyAttributesToCharRange(?)
 ## editorInfo.ace_setAttributeOnSelection(attribute, enabled)
 Sets an attribute on current range.
 Example: `call.editorInfo.ace_setAttributeOnSelection("turkey::balls", true); // turkey is the attribute here, balls is the value
 Notes: to remove the attribute pass enabled as false
 ## editorInfo.ace_toggleAttributeOnSelection(?)
 ## editorInfo.ace_getAttributeOnSelection(attribute, prevChar)
 Returns a boolean if an attribute exists on a selected range.
 prevChar value should be true if you want to get the previous Character attribute instead of the current selection for example
 if the caret is at position 0,1 (after first character) it's probable you want the attributes on the character at 0,0
 The attribute should be the string name of the attribute applied to the selection IE subscript
 Example usage: Apply the activeButton Class to a button if an attribute is on a highlighted/selected caret position or range.
 Example `var isItThere = documentAttributeManager.getAttributeOnSelection("turkey::balls", true);`

 See the ep_subscript plugin for an example of this function in action.
 Notes: Does not work on first or last character of a line.  Suffers from a race condition if called with aceEditEvent.
 ## editorInfo.ace_performSelectionChange(?)
 ## editorInfo.ace_doIndentOutdent(?)
 ## editorInfo.ace_doUndoRedo(?)
 ## editorInfo.ace_doInsertUnorderedList(?)
 ## editorInfo.ace_doInsertOrderedList(?)
 ## editorInfo.ace_performDocumentApplyAttributesToRange()

 ## editorInfo.ace_getAuthorInfos()
 Returns an info object about the author. Object key = author_id and info includes author's bg color value.
 Use to define your own authorship.
 ## editorInfo.ace_performDocumentReplaceRange(start, end, newText)
 This function replaces a range (from [x1,y1] to [x2,y2]) with `newText`.
 ## editorInfo.ace_performDocumentReplaceCharRange(startChar, endChar, newText)
 This function replaces a range (from y1 to y2) with `newText`.
 ## editorInfo.ace_renumberList(lineNum)
 If you delete a line, calling this method will fix the line numbering.
 ## editorInfo.ace_doReturnKey()
 Forces a return key at the current caret position.
 ## editorInfo.ace_isBlockElement(element)
 Returns true if your passed element is registered as a block element
 ## editorInfo.ace_getLineListType(lineNum)
 Returns the line's html list type.
 ## editorInfo.ace_caretLine()
 Returns X position of the caret.
 ## editorInfo.ace_caretColumn()
 Returns Y position of the caret.
 ## editorInfo.ace_caretDocChar()
 Returns the Y offset starting from [x=0,y=0]
 ## editorInfo.ace_isWordChar(?)
--- a/doc/api/embed_parameters.md
+++ b/doc/api/embed_parameters.md
@ -0,0 +1,68 @@
 # Embed parameters
 You can easily embed your etherpad-lite into any webpage by using iframes. You can configure the embedded pad using embed parameters.

 Example:

 Cut and paste the following code into any webpage to embed a pad. The parameters below will hide the chat and the line numbers.

 ```
 <iframe src='http://pad.test.de/p/PAD_NAME?showChat=false&showLineNumbers=false' width=600 height=400></iframe>
 ```

 ## showLineNumbers
 * Boolean

 Default: true

 ## showControls
 * Boolean

 Default: true

 ## showChat
 * Boolean

 Default: true

 ## useMonospaceFont
 * Boolean

 Default: false

 ## userName
 * String

 Default: "unnamed"

 Example: `userName=Etherpad%20User`

 ## userColor
 * String (css hex color value)

 Default: randomly chosen by pad server

 Example: `userColor=%23ff9900`

 ## noColors
 * Boolean

 Default: false

 ## alwaysShowChat
 * Boolean

 Default: false

 ## lang
 * String

 Default: en

 Example: `lang=ar` (translates the interface into Arabic)

 ## rtl
 * Boolean

 Default: true
 Displays pad text from right to left.

--- a/doc/api/hooks_client-side.md
+++ b/doc/api/hooks_client-side.md
@ -0,0 +1,389 @@
 # Client-side hooks
 Most of these hooks are called during or in order to set up the formatting process.

 ## documentReady
 Called from: src/templates/pad.html

 Things in context:

 nothing

 This hook proxies the functionality of jQuery's `$(document).ready` event.

 ## aceDomLinePreProcessLineAttributes
 Called from: src/static/js/domline.js

 Things in context:

 1. domline - The current DOM line being processed
 2. cls - The class of the current block element (useful for styling)

 This hook is called for elements in the DOM that have the "lineMarkerAttribute" set. You can add elements into this category with the aceRegisterBlockElements hook above.  This hook is run BEFORE the numbered and ordered lists logic is applied.

 The return value of this hook should have the following structure:

 `{ preHtml: String, postHtml: String, processedMarker: Boolean }`

 The preHtml and postHtml values will be added to the HTML display of the element, and if processedMarker is true, the engine won't try to process it any more.

 ## aceDomLineProcessLineAttributes
 Called from: src/static/js/domline.js

 Things in context:

 1. domline - The current DOM line being processed
 2. cls - The class of the current block element (useful for styling)

 This hook is called for elements in the DOM that have the "lineMarkerAttribute" set. You can add elements into this category with the aceRegisterBlockElements hook above.  This hook is run AFTER the ordered and numbered lists logic is applied.

 The return value of this hook should have the following structure:

 `{ preHtml: String, postHtml: String, processedMarker: Boolean }`

 The preHtml and postHtml values will be added to the HTML display of the element, and if processedMarker is true, the engine won't try to process it any more.

 ## aceCreateDomLine
 Called from: src/static/js/domline.js

 Things in context:

 1. domline - the current DOM line being processed
 2. cls - The class of the current element (useful for styling)

 This hook is called for any line being processed by the formatting engine, unless the aceDomLineProcessLineAttributes hook from above returned true, in which case this hook is skipped.

 The return value of this hook should have the following structure:

 `{ extraOpenTags: String, extraCloseTags: String, cls: String }`

 extraOpenTags and extraCloseTags will be added before and after the element in question, and cls will be the new class of the element going forward.

 ## acePostWriteDomLineHTML
 Called from: src/static/js/domline.js

 Things in context:

 1. node - the DOM node that just got written to the page

 This hook is for right after a node has been fully formatted and written to the page.

 ## aceAttribsToClasses
 Called from: src/static/js/linestylefilter.js

 Things in context:

 1. linestylefilter - the JavaScript object that's currently processing the ace attributes
 2. key - the current attribute being processed
 3. value - the value of the attribute being processed

 This hook is called during the attribute processing procedure, and should be used to translate key, value pairs into valid HTML classes that can be inserted into the DOM.

 The return value for this function should be a list of classes, which will then be parsed into a valid class string.

 ## aceAttribClasses
 Called from: src/static/js/linestylefilter.js

 Things in context:
 1. Attributes - Object of Attributes

 This hook is called when attributes are investigated on a line.  It is useful if you want to add another attribute type or property type to a pad.

 Example:
 ```
 exports.aceAttribClasses = function(hook_name, attr, cb){
  attr.sub = 'tag:sub';
  cb(attr);
 }
 ```

 ## aceGetFilterStack
 Called from: src/static/js/linestylefilter.js

 Things in context:

 1. linestylefilter - the JavaScript object that's currently processing the ace attributes
 2. browser - an object indicating which browser is accessing the page

 This hook is called to apply custom regular expression filters to a set of styles. The one example available is the ep_linkify plugin, which adds internal links. They use it to find the telltale `[[ ]]` syntax that signifies internal links, and finding that syntax, they add in the internalHref attribute to be later used by the aceCreateDomLine hook (documented above).

 ## aceEditorCSS
 Called from: src/static/js/ace.js

 Things in context: None

 This hook is provided to allow custom CSS files to be loaded. The return value should be an array of resource urls or paths relative to the plugins directory.

 ## aceInitInnerdocbodyHead
 Called from: src/static/js/ace.js

 Things in context:

 1. iframeHTML - the HTML of the editor iframe up to this point, in array format

 This hook is called during the creation of the editor HTML. The array should have lines of HTML added to it, giving the plugin author a chance to add in meta, script, link, and other tags that go into the `<head>` element of the editor HTML document.

 ## aceEditEvent
 Called from: src/static/js/ace2_inner.js

 Things in context:

 1. callstack - a bunch of information about the current action
 2. editorInfo - information about the user who is making the change
 3. rep - information about where the change is being made
 4. documentAttributeManager - information about attributes in the document (this is a mystery to me)

 This hook is made available to edit the edit events that might occur when changes are made. Currently you can change the editor information, some of the meanings of the edit, and so on. You can also make internal changes (internal to your plugin) that use the information provided by the edit event.

 ## aceRegisterNonScrollableEditEvents
 Called from: src/static/js/ace2_inner.js

 Things in context: None

 When aceEditEvent (documented above) finishes processing the event, it scrolls the viewport to make caret visible to the user, but if you don't want that behavior to happen you can use this hook to register which edit events should not scroll viewport. The return value of this hook should be a list of event names.

 Example:
 ```
 exports.aceRegisterNonScrollableEditEvents = function(){
  return [ 'repaginate', 'updatePageCount' ];
 }
 ```

 ## aceRegisterBlockElements
 Called from: src/static/js/ace2_inner.js

 Things in context: None

 The return value of this hook will add elements into the "lineMarkerAttribute" category, making the aceDomLineProcessLineAttributes hook (documented below) call for those elements.

 ## aceInitialized
 Called from: src/static/js/ace2_inner.js

 Things in context:

 1. editorInfo - information about the user who will be making changes through the interface, and a way to insert functions into the main ace object (see ep_headings)
 2. rep - information about where the user's cursor is
 3. documentAttributeManager - some kind of magic

 This hook is for inserting further information into the ace engine, for later use in formatting hooks.

 ## postAceInit
 Called from: src/static/js/pad.js

 Things in context:

 1. ace - the ace object that is applied to this editor.
 2. pad - the pad object of the current pad.

 ## postToolbarInit
 Called from: src/static/js/pad_editbar.js

 Things in context:

 1. ace - the ace object that is applied to this editor.
 2. toolbar - Editbar instance. See below for the Editbar documentation.

 Can be used to register custom actions to the toolbar.

 Usage examples:

 * [https://github.com/tiblu/ep_authorship_toggle]()

 ## postTimesliderInit
 Called from: src/static/js/timeslider.js

 There doesn't appear to be any example available of this particular hook being used, but it gets fired after the timeslider is all set up.

 ## goToRevisionEvent
 Called from: src/static/js/broadcast.js

 Things in context:

 1. rev - The newRevision

 This hook gets fired both on timeslider load (as timeslider shows a new revision) and when the new revision is showed to a user.
 There doesn't appear to be any example available of this particular hook being used.

 ## userJoinOrUpdate
 Called from: src/static/js/pad_userlist.js

 Things in context:

 1. info - the user information

 This hook is called on the client side whenever a user joins or changes. This can be used to create notifications or an alternate user list.

 ## chatNewMessage
 Called from: src/static/js/chat.js

 Things in context:

 1. authorName - The user that wrote this message
 2. author - The authorID of the user that wrote the message
 3. text - the message text
 4. sticky (boolean) - if you want the gritter notification bubble to fade out on its own or just sit there
 5. timestamp - the timestamp of the chat message
 6. timeStr - the timestamp as a formatted string
 7. duration - for how long in milliseconds should the gritter notification appear (0 to disable)

 This hook is called on the client side whenever a chat message is received from the server. It can be used to create different notifications for chat messages.

 ## collectContentPre
 Called from: src/static/js/contentcollector.js

 Things in context:

 1. cc - the contentcollector object
 2. state - the current state of the change being made
 3. tname - the tag name of this node currently being processed
 4. styl - the style applied to the node (probably CSS) -- Note the typo
 5. cls - the HTML class string of the node

 This hook is called before the content of a node is collected by the usual methods. The cc object can be used to do a bunch of things that modify the content of the pad. See, for example, the heading1 plugin for etherpad original.

 E.g. if you need to apply an attribute to newly inserted characters,
 call cc.doAttrib(state, "attributeName") which results in an attribute attributeName=true.

 If you want to specify also a value, call cc.doAttrib(state, "attributeName::value")
 which results in an attribute attributeName=value.


 ## collectContentImage
 Called from: src/static/js/contentcollector.js

 Things in context:

 1. cc - the contentcollector object
 2. state - the current state of the change being made
 3. tname - the tag name of this node currently being processed
 4. style - the style applied to the node (probably CSS)
 5. cls - the HTML class string of the node
 6. node - the node being modified

 This hook is called before the content of an image node is collected by the usual methods. The cc object can be used to do a bunch of things that modify the content of the pad.

 Example:

 ```
 exports.collectContentImage = function(name, context){
  context.state.lineAttributes.img = context.node.outerHTML;
 }

 ```

 ## collectContentPost
 Called from: src/static/js/contentcollector.js

 Things in context:

 1. cc - the contentcollector object
 2. state - the current state of the change being made
 3. tname - the tag name of this node currently being processed
 4. style - the style applied to the node (probably CSS)
 5. cls - the HTML class string of the node

 This hook is called after the content of a node is collected by the usual methods. The cc object can be used to do a bunch of things that modify the content of the pad. See, for example, the heading1 plugin for etherpad original.

 ## handleClientMessage_`name`
 Called from: `src/static/js/collab_client.js`

 Things in context:

 1. payload - the data that got sent with the message (use it for custom message content)

 This hook gets called every time the client receives a message of type `name`. This can most notably be used with the new HTTP API call, "sendClientsMessage", which sends a custom message type to all clients connected to a pad. You can also use this to handle existing types.

 `collab_client.js` has a pretty extensive list of message types, if you want to take a look.

 ##aceStartLineAndCharForPoint-aceEndLineAndCharForPoint
 Called from: src/static/js/ace2_inner.js

 Things in context:

 1. callstack - a bunch of information about the current action
 2. editorInfo - information about the user who is making the change
 3. rep - information about where the change is being made
 4. root - the span element of the current line
 5. point - the starting/ending element where the cursor highlights
 6. documentAttributeManager - information about attributes in the document

 This hook is provided to allow a plugin to turn DOM node selection into [line,char] selection.
 The return value should be an array of [line,char]

 ##aceKeyEvent
 Called from: src/static/js/ace2_inner.js

 Things in context:

 1. callstack - a bunch of information about the current action
 2. editorInfo - information about the user who is making the change
 3. rep - information about where the change is being made
 4. documentAttributeManager - information about attributes in the document
 5. evt - the fired event

 This hook is provided to allow a plugin to handle key events.
 The return value should be true if you have handled the event.

 ##collectContentLineText
 Called from: src/static/js/contentcollector.js

 Things in context:

 1. cc - the contentcollector object
 2. state - the current state of the change being made
 3. tname - the tag name of this node currently being processed
 4. text - the text for that line

 This hook allows you to validate/manipulate the text before it's sent to the server side.
 The return value should be the validated/manipulated text.

 ##collectContentLineBreak
 Called from: src/static/js/contentcollector.js

 Things in context:

 1. cc - the contentcollector object
 2. state - the current state of the change being made
 3. tname - the tag name of this node currently being processed

 This hook is provided to allow whether the br tag should induce a new magic domline or not.
 The return value should be either true(break the line) or false.

 ##disableAuthorColorsForThisLine
 Called from: src/static/js/linestylefilter.js

 Things in context:

 1. linestylefilter - the JavaScript object that's currently processing the ace attributes
 2. text - the line text
 3. class - line class

 This hook is provided to allow whether a given line should be deliniated with multiple authors.
 Multiple authors in one line cause the creation of magic span lines. This might not suit you and
 now you can disable it and handle your own deliniation.
 The return value should be either true(disable) or false.

 ## aceSetAuthorStyle
 Called from: src/static/js/ace2_inner.js

 Things in context:

 1. dynamicCSS - css manager for inner ace
 2. outerDynamicCSS - css manager for outer ace
 3. parentDynamicCSS - css manager for parent document
 4. info - author style info
 5. author - author info
 6. authorSelector - css selector for author span in inner ace

 This hook is provided to allow author highlight style to be modified.
 Registered hooks should return 1 if the plugin handles highlighting.  If no plugin returns 1, the core will use the default background-based highlighting.

 ## aceSelectionChanged
 Called from: src/static/js/ace2_inner.js

 Things in context:

 1. rep - information about where the user's cursor is
 2. documentAttributeManager - information about attributes in the document

 This hook allows a plugin to react to a cursor or selection change,
 perhaps to update a UI element based on the style at the cursor location.
--- a/doc/api/hooks_overview.md
+++ b/doc/api/hooks_overview.md
@ -0,0 +1,117 @@
 # Hooks

 A hook function is registered with a hook via the plugin's `ep.json` file. See
 the Plugins section for details. A hook may have many registered functions from
 different plugins.

 Some hooks call their registered functions one at a time until one of them
 returns a value. Others always call all of their registered functions and
 combine the results (if applicable).

 ## Registered hook functions

 Note: The documentation in this section applies to every hook unless the
 hook-specific documentation says otherwise.

 ### Arguments

 Hook functions are called with three arguments:

 1. `hookName` - The name of the hook being invoked.
 2. `context` - An object with some relevant information about the context of the
   call. See the hook-specific documentation for details.
 3. `cb` - For asynchronous operations this callback can be called to signal
   completion and optionally provide a return value. The callback takes a single
   argument, the meaning of which depends on the hook (see the "Return values"
   section for general information that applies to most hooks). This callback
   always returns `undefined`.

 ### Expected behavior

 The presence of a callback parameter suggests that every hook function can run
 asynchronously. While that is the eventual goal, there are some legacy hooks
 that expect their hook functions to provide a value synchronously. For such
 hooks, the hook functions must do one of the following:

 * Call the callback with a non-Promise value (`undefined` is acceptable) and
  return `undefined`, in that order.
 * Return a non-Promise value other than `undefined` (`null` is acceptable) and
  never call the callback. Note that `async` functions *always* return a
  Promise, so they must never be used for synchronous hooks.
 * Only have two parameters (`hookName` and `context`) and return any non-Promise
  value (`undefined` is acceptable).

 For hooks that permit asynchronous behavior, the hook functions must do one or
 more of the following:

 * Return `undefined` and call the callback, in either order.
 * Return something other than `undefined` (`null` is acceptable) and never call
  the callback. Note that `async` functions *always* return a Promise, so they
  must never call the callback.
 * Only have two parameters (`hookName` and `context`).

 Note that the acceptable behaviors for asynchronous hook functions is a superset
 of the acceptable behaviors for synchronous hook functions.

 WARNING: The number of parameters is determined by examining
 [Function.length](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/length),
 which does not count [default
 parameters](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Default_parameters)
 or ["rest"
 parameters](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/rest_parameters).
 To avoid problems, do not use default or rest parameters when defining hook
 functions.

 ### Return values

 A hook function can provide a value to Etherpad in one of the following ways:

 * Pass the desired value as the first argument to the callback.
 * Return the desired value directly. The value must not be `undefined` unless
  the hook function only has two parameters. (Hook functions with three
  parameters that want to provide `undefined` should instead use the callback.)
 * For hooks that permit asynchronous behavior, return a Promise that resolves to
  the desired value.
 * For hooks that permit asynchronous behavior, pass a Promise that resolves to
  the desired value as the first argument to the callback.

 Examples:

 ```javascript
 exports.exampleOne = (hookName, context, callback) => {
  return 'valueOne';
 };

 exports.exampleTwo = (hookName, context, callback) => {
  callback('valueTwo');
  return;
 };

 // ONLY FOR HOOKS THAT PERMIT ASYNCHRONOUS BEHAVIOR
 exports.exampleThree = (hookName, context, callback) => {
  return new Promise('valueThree');
 };

 // ONLY FOR HOOKS THAT PERMIT ASYNCHRONOUS BEHAVIOR
 exports.exampleFour = (hookName, context, callback) => {
  callback(new Promise('valueFour'));
  return;
 };

 // ONLY FOR HOOKS THAT PERMIT ASYNCHRONOUS BEHAVIOR
 exports.exampleFive = async (hookName, context) => {
  // Note that this function is async, so it actually returns a Promise that
  // is resolved to 'valueFive'.
  return 'valueFive';
 };
 ```

 Etherpad collects the values provided by the hook functions into an array,
 filters out all `undefined` values, then flattens the array one level.
 Flattening one level makes it possible for a hook function to behave as if it
 were multiple separate hook functions.

 For example: Suppose a hook has eight registered functions that return the
 following values: `1`, `[2]`, `['3a', '3b']` `[[4]]`, `undefined`,
 `[undefined]`, `[]`, and `null`. The value returned to the caller of the hook is
 `[1, 2, '3a', '3b', [4], undefined, null]`.
--- a/doc/api/hooks_server-side.md
+++ b/doc/api/hooks_server-side.md
@ -0,0 +1,842 @@
 # Server-side hooks
 These hooks are called on server-side.

 ## loadSettings
 Called from: src/node/server.js

 Things in context:

 1. settings - the settings object

 Use this hook to receive the global settings in your plugin.

 ## shutdown
 Called from: src/node/server.js

 Things in context: None

 This hook runs before shutdown. Use it to stop timers, close sockets and files,
 flush buffers, etc. The database is not available while this hook is running.
 The shutdown function must not block for long because there is a short timeout
 before the process is forcibly terminated.

 The shutdown function must return a Promise, which must resolve to `undefined`.
 Returning `callback(value)` will return a Promise that is resolved to `value`.

 Example:

 ```
 // using an async function
 exports.shutdown = async (hookName, context) => {
  await flushBuffers();
 };
 ```

 ## pluginUninstall
 Called from: src/static/js/pluginfw/installer.js

 Things in context:

 1. plugin_name - self-explanatory

 If this hook returns an error, the callback to the uninstall function gets an error as well. This mostly seems useful for handling additional features added in based on the installation of other plugins, which is pretty cool!

 ## pluginInstall
 Called from: src/static/js/pluginfw/installer.js

 Things in context:

 1. plugin_name - self-explanatory

 If this hook returns an error, the callback to the install function gets an error, too. This seems useful for adding in features when a particular plugin is installed.

 ## init_`<plugin name>`
 Called from: src/static/js/pluginfw/plugins.js

 Things in context: None

 This function is called after a specific plugin is initialized. This would probably be more useful than the previous two functions if you only wanted to add in features to one specific plugin.

 ## expressConfigure
 Called from: src/node/hooks/express.js

 Things in context:

 1. app - the main application object

 This is a helpful hook for changing the behavior and configuration of the application. It's called right after the application gets configured.

 ## expressCreateServer
 Called from: src/node/hooks/express.js

 Things in context:

 1. app - the main express application object (helpful for adding new paths and such)
 2. server - the http server object

 This hook gets called after the application object has been created, but before it starts listening. This is similar to the expressConfigure hook, but it's not guaranteed that the application object will have all relevant configuration variables.

 ## expressCloseServer

 Called from: src/node/hooks/express.js

 Things in context: Nothing

 This hook is called when the HTTP server is closing, which happens during
 shutdown (see the shutdown hook) and when the server restarts (e.g., when a
 plugin is installed via the `/admin/plugins` page). The HTTP server may or may
 not already be closed when this hook executes.

 Example:

 ```
 exports.expressCloseServer = async () => {
  await doSomeCleanup();
 };
 ```

 ## eejsBlock_`<name>`
 Called from: src/node/eejs/index.js

 Things in context:

 1. content - the content of the block

 This hook gets called upon the rendering of an ejs template block. For any specific kind of block, you can change how that block gets rendered by modifying the content object passed in.

 Available blocks in `pad.html` are:

 * `htmlHead` - after `<html>` and immediately before the title tag
 * `styles` - the style `<link>`s
 * `body` - the contents of the body tag
 * `editbarMenuLeft` - the left tool bar (consider using the toolbar controller instead of manually adding html here)
 * `editbarMenuRight` - right tool bar
 * `afterEditbar` - allows you to add stuff immediately after the toolbar
 * `userlist` - the contents of the userlist dropdown
 * `loading` - the initial loading message
 * `mySettings` - the left column of the settings dropdown ("My view"); intended for adding checkboxes only
 * `mySettings.dropdowns` - add your dropdown settings here
 * `globalSettings` - the right column of the settings dropdown ("Global view")
 * `importColumn` - import form
 * `exportColumn` - export form
 * `modals` - Contains all connectivity messages
 * `embedPopup` - the embed dropdown
 * `scripts` - Add your script tags here, if you really have to (consider use client-side hooks instead)

 `timeslider.html` blocks:

 * `timesliderStyles`
 * `timesliderScripts`
 * `timesliderBody`
 * `timesliderTop`
 * `timesliderEditbarRight`
 * `modals`

 `index.html` blocks:

 * `indexCustomStyles` - contains the `index.css` `<link>` tag, allows you to add your own or to customize the one provided by the active skin
 * `indexWrapper` - contains the form for creating new pads
 * `indexCustomScripts` - contains the `index.js` `<script>` tag, allows you to add your own or to customize the one provided by the active skin

 ## padInitToolbar
 Called from: src/node/hooks/express/specialpages.js

 Things in context:

 1. toolbar - the toolbar controller that will render the toolbar eventually

 Here you can add custom toolbar items that will be available in the toolbar config in `settings.json`. For more about the toolbar controller see the API section.

 Usage examples:

 * https://github.com/tiblu/ep_authorship_toggle

 ## onAccessCheck
 Called from: src/node/db/SecurityManager.js

 Things in context:

 1. padID - the pad the user wants to access
 2. token - the token of the author
 3. sessionCookie - the session the use has

 This hook gets called when the access to the concrete pad is being checked. Return `false` to deny access.

 ## padCreate
 Called from: src/node/db/Pad.js

 Things in context:

 1. pad - the pad instance
 2. author - the id of the author who created the pad

 This hook gets called when a new pad was created.

 ## padLoad
 Called from: src/node/db/Pad.js

 Things in context:

 1. pad - the pad instance

 This hook gets called when a pad was loaded. If a new pad was created and loaded this event will be emitted too.

 ## padUpdate
 Called from: src/node/db/Pad.js

 Things in context:

 1. pad - the pad instance
 2. author - the id of the author who updated the pad
 3. revs - the index of the new revision
 4. changeset - the changeset of this revision (see [Changeset Library](#index_changeset_library))

 This hook gets called when an existing pad was updated.

 ## padCopy
 Called from: src/node/db/Pad.js

 Things in context:

 1. originalPad - the source pad instance
 2. destinationID - the id of the pad copied from originalPad

 This hook gets called when an existing pad was copied.

 Usage examples:

 * https://github.com/ether/ep_comments

 ## padRemove
 Called from: src/node/db/Pad.js

 Things in context:

 1. padID

 This hook gets called when an existing pad was removed/deleted.

 Usage examples:

 * https://github.com/ether/ep_comments

 ## socketio
 Called from: src/node/hooks/express/socketio.js

 Things in context:

 1. app - the application object
 2. io - the socketio object
 3. server - the http server object

 I have no idea what this is useful for, someone else will have to add this description.

 ## preAuthorize
 Called from: src/node/hooks/express/webaccess.js

 Things in context:

 1. req - the request object
 2. res - the response object
 3. next - bypass callback. If this is called instead of the normal callback then
   all remaining access checks are skipped.

 This hook is called for each HTTP request before any authentication checks are
 performed. Example uses:

 * Always grant access to static content.
 * Process an OAuth callback.
 * Drop requests from IP addresses that have failed N authentication checks
  within the past X minutes.

 A preAuthorize function is always called for each request unless a preAuthorize
 function from another plugin (if any) has already explicitly granted or denied
 the request.

 You can pass the following values to the provided callback:

 * `[]` defers the access decision to the normal authentication and authorization
  checks (or to a preAuthorize function from another plugin, if one exists).
 * `[true]` immediately grants access to the requested resource, unless the
  request is for an `/admin` page in which case it is treated the same as `[]`.
  (This prevents buggy plugins from accidentally granting admin access to the
  general public.)
 * `[false]` immediately denies the request. The preAuthnFailure hook will be
  called to handle the failure.

 Example:

 ```
 exports.preAuthorize = (hookName, context, cb) => {
  if (ipAddressIsFirewalled(context.req)) return cb([false]);
  if (requestIsForStaticContent(context.req)) return cb([true]);
  if (requestIsForOAuthCallback(context.req)) return cb([true]);
  return cb([]);
 };
 ```

 ## authorize
 Called from: src/node/hooks/express/webaccess.js

 Things in context:

 1. req - the request object
 2. res - the response object
 3. next - ?
 4. resource - the path being accessed

 This hook is called to handle authorization. It is especially useful for
 controlling access to specific paths.

 A plugin's authorize function is only called if all of the following are true:

 * The request is not for static content or an API endpoint. (Requests for static
  content and API endpoints are always authorized, even if unauthenticated.)
 * The `requireAuthentication` and `requireAuthorization` settings are both true.
 * The user has already successfully authenticated.
 * The user is not an admin (admin users are always authorized).
 * The path being accessed is not an `/admin` path (`/admin` paths can only be
  accessed by admin users, and admin users are always authorized).
 * An authorize function from a different plugin has not already caused
  authorization to pass or fail.

 Note that the authorize hook cannot grant access to `/admin` pages. If admin
 access is desired, the `is_admin` user setting must be set to true. This can be
 set in the settings file or by the authenticate hook.

 You can pass the following values to the provided callback:

 * `[true]` or `['create']` will grant access to modify or create the pad if the
  request is for a pad, otherwise access is simply granted. Access to a pad will
  be downgraded to modify-only if `settings.editOnly` is true or the user's
  `canCreate` setting is set to `false`, and downgraded to read-only if the
  user's `readOnly` setting is `true`.
 * `['modify']` will grant access to modify but not create the pad if the request
  is for a pad, otherwise access is simply granted. Access to a pad will be
  downgraded to read-only if the user's `readOnly` setting is `true`.
 * `['readOnly']` will grant read-only access.
 * `[false]` will deny access.
 * `[]` or `undefined` will defer the authorization decision to the next
  authorization plugin (if any, otherwise deny).

 Example:

 ```
 exports.authorize = (hookName, context, cb) => {
  const user = context.req.session.user;
  const path = context.req.path;  // or context.resource
  if (isExplicitlyProhibited(user, path)) return cb([false]);
  if (isExplicitlyAllowed(user, path)) return cb([true]);
  return cb([]);  // Let the next authorization plugin decide
 };
 ```

 ## authenticate
 Called from: src/node/hooks/express/webaccess.js

 Things in context:

 1. req - the request object
 2. res - the response object
 3. users - the users object from settings.json (possibly modified by plugins)
 4. next - ?
 5. username - the username used (optional)
 6. password - the password used (optional)

 This hook is called to handle authentication.

 Plugins that supply an authenticate function should probably also supply an
 authnFailure function unless falling back to HTTP basic authentication is
 appropriate upon authentication failure.

 This hook is only called if either the `requireAuthentication` setting is true
 or the request is for an `/admin` page.

 Calling the provided callback with `[true]` or `[false]` will cause
 authentication to succeed or fail, respectively. Calling the callback with `[]`
 or `undefined` will defer the authentication decision to the next authentication
 plugin (if any, otherwise fall back to HTTP basic authentication).

 If you wish to provide a mix of restricted and anonymous access (e.g., some pads
 are private, others are public), you can "authenticate" (as a guest account)
 users that have not yet logged in, and rely on other hooks (e.g., authorize,
 onAccessCheck, handleMessageSecurity) to authorize specific privileged actions.

 If authentication is successful, the authenticate function MUST set
 `context.req.session.user` to the user's settings object. The `username`
 property of this object should be set to the user's username. The settings
 object should come from global settings (`context.users[username]`).

 Example:

 ```
 exports.authenticate = (hook_name, context, cb) => {
  if (notApplicableToThisPlugin(context)) {
    return cb([]);  // Let the next authentication plugin decide
  }
  const username = authenticate(context);
  if (!username) {
    console.warn(`ep_myplugin.authenticate: Failed authentication from IP ${context.req.ip}`);
    return cb([false]);
  }
  console.info(`ep_myplugin.authenticate: Successful authentication from IP  $c o n t e x t . r e q . i p f o r u s e r$ {username}`);
  const users = context.users;
  if (!(username in users)) users[username] = {};
  users[username].username = username;
  context.req.session.user = users[username];
  return cb([true]);
 };
 ```

 ## authFailure
 Called from: src/node/hooks/express/webaccess.js

 Things in context:

 1. req - the request object
 2. res - the response object
 3. next - ?

 **DEPRECATED:** Use authnFailure or authzFailure instead.

 This hook is called to handle an authentication or authorization failure.

 Plugins that supply an authenticate function should probably also supply an
 authnFailure function unless falling back to HTTP basic authentication is
 appropriate upon authentication failure.

 A plugin's authFailure function is only called if all of the following are true:

 * There was an authentication or authorization failure.
 * The failure was not already handled by an authFailure function from another
  plugin.
 * For authentication failures: The failure was not already handled by the
  authnFailure hook.
 * For authorization failures: The failure was not already handled by the
  authzFailure hook.

 Calling the provided callback with `[true]` tells Etherpad that the failure was
 handled and no further error handling is required. Calling the callback with
 `[]` or `undefined` defers error handling to the next authFailure plugin (if
 any, otherwise fall back to HTTP basic authentication for an authentication
 failure or a generic 403 page for an authorization failure).

 Example:

 ```
 exports.authFailure = (hookName, context, cb) => {
  if (notApplicableToThisPlugin(context)) {
    return cb([]);  // Let the next plugin handle the error
  }
  context.res.redirect(makeLoginURL(context.req));
  return cb([true]);
 };
 ```

 ## preAuthzFailure
 Called from: src/node/hooks/express/webaccess.js

 Things in context:

 1. req - the request object
 2. res - the response object

 This hook is called to handle a pre-authentication authorization failure.

 A plugin's preAuthzFailure function is only called if the pre-authentication
 authorization failure was not already handled by a preAuthzFailure function from
 another plugin.

 Calling the provided callback with `[true]` tells Etherpad that the failure was
 handled and no further error handling is required. Calling the callback with
 `[]` or `undefined` defers error handling to a preAuthzFailure function from
 another plugin (if any, otherwise fall back to a generic 403 error page).

 Example:

 ```
 exports.preAuthzFailure = (hookName, context, cb) => {
  if (notApplicableToThisPlugin(context)) return cb([]);
  context.res.status(403).send(renderFancy403Page(context.req));
  return cb([true]);
 };
 ```

 ## authnFailure
 Called from: src/node/hooks/express/webaccess.js

 Things in context:

 1. req - the request object
 2. res - the response object

 This hook is called to handle an authentication failure.

 Plugins that supply an authenticate function should probably also supply an
 authnFailure function unless falling back to HTTP basic authentication is
 appropriate upon authentication failure.

 A plugin's authnFailure function is only called if the authentication failure
 was not already handled by an authnFailure function from another plugin.

 Calling the provided callback with `[true]` tells Etherpad that the failure was
 handled and no further error handling is required. Calling the callback with
 `[]` or `undefined` defers error handling to an authnFailure function from
 another plugin (if any, otherwise fall back to the deprecated authFailure hook).

 Example:

 ```
 exports.authnFailure = (hookName, context, cb) => {
  if (notApplicableToThisPlugin(context)) return cb([]);
  context.res.redirect(makeLoginURL(context.req));
  return cb([true]);
 };
 ```

 ## authzFailure
 Called from: src/node/hooks/express/webaccess.js

 Things in context:

 1. req - the request object
 2. res - the response object

 This hook is called to handle a post-authentication authorization failure.

 A plugin's authzFailure function is only called if the authorization failure was
 not already handled by an authzFailure function from another plugin.

 Calling the provided callback with `[true]` tells Etherpad that the failure was
 handled and no further error handling is required. Calling the callback with
 `[]` or `undefined` defers error handling to an authzFailure function from
 another plugin (if any, otherwise fall back to the deprecated authFailure hook).

 Example:

 ```
 exports.authzFailure = (hookName, context, cb) => {
  if (notApplicableToThisPlugin(context)) return cb([]);
  if (needsPremiumAccount(context.req) && !context.req.session.user.premium) {
    context.res.status(200).send(makeUpgradeToPremiumAccountPage(context.req));
    return cb([true]);
  }
  // Use the generic 403 forbidden response.
  return cb([]);
 };
 ```

 ## handleMessage
 Called from: src/node/handler/PadMessageHandler.js

 Things in context:

 1. message - the message being handled
 2. socket - the socket.io Socket object
 3. client - **deprecated** synonym of socket

 This hook allows plugins to drop or modify incoming socket.io messages from
 clients, before Etherpad processes them.

 The handleMessage function must return a Promise. If the Promise resolves to
 `null`, the message is dropped. Returning `callback(value)` will return a
 Promise that is resolved to `value`.

 Examples:

 ```
 // Using an async function:
 exports.handleMessage = async (hookName, {message, socket}) => {
  if (message.type === 'USERINFO_UPDATE') {
    // Force the display name to the name associated with the account.
    const user = socket.client.request.session.user || {};
    if (user.name) message.data.userInfo.name = user.name;
  }
 };

 // Using a regular function:
 exports.handleMessage = (hookName, {message, socket}, callback) => {
  if (message.type === 'USERINFO_UPDATE') {
    // Force the display name to the name associated with the account.
    const user = socket.client.request.session.user || {};
    if (user.name) message.data.userInfo.name = user.name;
  }
  return callback();
 };
 ```

 ## handleMessageSecurity
 Called from: src/node/handler/PadMessageHandler.js

 Things in context:

 1. message - the message being handled
 2. socket - the socket.io Socket object
 3. client - **deprecated** synonym of socket

 This hook allows plugins to grant temporary write access to a pad. It is called
 for each incoming message from a client. If write access is granted, it applies
 to the current message and all future messages from the same socket.io
 connection until the next `CLIENT_READY` or `SWITCH_TO_PAD` message. Read-only
 access is reset **after** each `CLIENT_READY` or `SWITCH_TO_PAD` message, so
 granting write access has no effect for those message types.

 The handleMessageSecurity function must return a Promise. If the Promise
 resolves to `true`, write access is granted as described above. Returning
 `callback(value)` will return a Promise that is resolved to `value`.

 Examples:

 ```
 // Using an async function:
 exports.handleMessageSecurity = async (hookName, {message, socket}) => {
  if (shouldGrantWriteAccess(message, socket)) return true;
  return;
 };

 // Using a regular function:
 exports.handleMessageSecurity = (hookName, {message, socket}, callback) => {
  if (shouldGrantWriteAccess(message, socket)) return callback(true);
  return callback();
 };
 ```

 ## clientVars
 Called from: src/node/handler/PadMessageHandler.js

 Things in context:

 1. clientVars - the basic `clientVars` built by the core
 2. pad - the pad this session is about
 3. socket - the socket.io Socket object

 This hook is called after a client connects but before the initial configuration
 is sent to the client. Plugins can use this hook to manipulate the
 configuration. (Example: Add a tracking ID for an external analytics tool that
 is used client-side.)

 The clientVars function must return a Promise that resolves to an object (or
 null/undefined) whose properties will be merged into `context.clientVars`.
 Returning `callback(value)` will return a Promise that is resolved to `value`.

 You can modify `context.clientVars` to change the values sent to the client, but
 beware: async functions from other clientVars plugins might also be reading or
 manipulating the same `context.clientVars` object. For this reason it is
 recommended you return an object rather than modify `context.clientVars`.

 If needed, you can access the user's account information (if authenticated) via
 `context.socket.client.request.session.user`.

 Examples:

 ```
 // Using an async function
 exports.clientVars = async (hookName, context) => {
  const user = context.socket.client.request.session.user || {};
  return {'accountUsername': user.username || '<unknown>'}
 };

 // Using a regular function
 exports.clientVars = (hookName, context, callback) => {
  const user = context.socket.client.request.session.user || {};
  return callback({'accountUsername': user.username || '<unknown>'});
 };
 ```

 This can be accessed on the client-side using `clientVars.currentYear`.

 ## getLineHTMLForExport
 Called from: src/node/utils/ExportHtml.js

 Things in context:

 1. apool - pool object
 2. attribLine - line attributes
 3. text - line text

 This hook will allow a plug-in developer to re-write each line when exporting to HTML.

 Example:
 ```
 var Changeset = require("ep_etherpad-lite/static/js/Changeset");

 exports.getLineHTMLForExport = function (hook, context) {
  var header = _analyzeLine(context.attribLine, context.apool);
  if (header) {
    return "<" + header + ">" + context.lineContent + "</" + header + ">";
  }
 }

 function _analyzeLine(alineAttrs, apool) {
  var header = null;
  if (alineAttrs) {
    var opIter = Changeset.opIterator(alineAttrs);
    if (opIter.hasNext()) {
      var op = opIter.next();
      header = Changeset.opAttributeValue(op, 'heading', apool);
    }
  }
  return header;
 }
 ```

 ## exportHTMLAdditionalContent
 Called from: src/node/utils/ExportHtml.js

 Things in context:

 1. padId

 This hook will allow a plug-in developer to include additional HTML content in
 the body of the exported HTML.

 Example:

 ```
 exports.exportHTMLAdditionalContent = async (hookName, {padId}) => {
  return 'I am groot in ' + padId;
 };
 ```

 ## stylesForExport
 Called from: src/node/utils/ExportHtml.js

 Things in context:

 1. padId - The Pad Id

 This hook will allow a plug-in developer to append Styles to the Exported HTML.

 Example:

 ```
 exports.stylesForExport = function(hook, padId, cb){
  cb("body{font-size:13.37em !important}");
 }
 ```

 ## aceAttribClasses
 Called from: src/static/js/linestylefilter.js

 This hook is called when attributes are investigated on a line. It is useful if
 you want to add another attribute type or property type to a pad.

 An attributes object is passed to the aceAttribClasses hook functions instead of
 the usual context object. A hook function can either modify this object directly
 or provide an object whose properties will be assigned to the attributes object.

 Example:

 ```
 exports.aceAttribClasses = (hookName, attrs, cb) => {
  return cb([{
    sub: 'tag:sub',
  }]);
 };
 ```

 ## exportFileName
 Called from src/node/handler/ExportHandler.js

 Things in context:

 1. padId

 This hook will allow a plug-in developer to modify the file name of an exported pad.  This is useful if you want to export a pad under another name and/or hide the padId under export.  Note that the doctype or file extension cannot be modified for security reasons.

 Example:

 ```
 exports.exportFileName = function(hook, padId, callback){
  callback("newFileName"+padId);
 }
 ```

 ## exportHtmlAdditionalTags
 Called from src/node/utils/ExportHtml.js

 Things in context:

 1. Pad object

 This hook will allow a plug-in developer to include more properties and attributes to support during HTML Export. If tags are stored as `['color', 'red']` on the attribute pool, use `exportHtmlAdditionalTagsWithData` instead. An Array should be returned.

 Example:
 ```
 // Add the props to be supported in export
 exports.exportHtmlAdditionalTags = function(hook, pad, cb){
  var padId = pad.id;
  cb(["massive","jugs"]);
 };
 ```

 ## exportHtmlAdditionalTagsWithData
 Called from src/node/utils/ExportHtml.js

 Things in context:

 1. Pad object

 Identical to `exportHtmlAdditionalTags`, but for tags that are stored with a specific value (not simply `true`) on the attribute pool. For example `['color', 'red']`, instead of `['bold', true]`. This hook will allow a plug-in developer to include more properties and attributes to support during HTML Export. An Array of arrays should be returned. The exported HTML will contain tags like `<span data-color="red">` for the content where attributes are `['color', 'red']`.

 Example:
 ```
 // Add the props to be supported in export
 exports.exportHtmlAdditionalTagsWithData = function(hook, pad, cb){
  var padId = pad.id;
  cb([["color", "red"], ["color", "blue"]]);
 };
 ```

 ## exportEtherpadAdditionalContent
 Called from src/node/utils/ExportEtherpad.js and
 src/node/utils/ImportEtherpad.js

 Things in context: Nothing

 Useful for exporting and importing pad metadata that is stored in the database
 but not in the pad's content or attributes. For example, in ep_comments_page the
 comments are stored as `comments:padId:uniqueIdOfComment` so a complete export
 of all pad data to an `.etherpad` file must include the `comments:padId:*`
 records.

 Example:

 ```
 // Add support for exporting comments metadata
 exports.exportEtherpadAdditionalContent = () => ['comments'];
 ```

 ## userLeave
 Called from src/node/handler/PadMessageHandler.js

 This in context:

 1. session (including the pad id and author id)

 This hook gets called when an author leaves a pad. This is useful if you want to perform certain actions after a pad has been edited

 Example:

 ```
 exports.userLeave = function(hook, session, callback) {
  console.log('%s left pad %s', session.author, session.padId);
 };
 ```

 ### clientReady
 Called from src/node/handler/PadMessageHandler.js

 This in context:

 1. message

 This hook gets called when handling a CLIENT_READY which is the first message from the client to the server.

 Example:

 ```
 exports.clientReady = function(hook, message) {
  console.log('Client has entered the pad' + message.padId);
 };
 ```
--- a/doc/api/http_api.md
+++ b/doc/api/http_api.md
@ -0,0 +1,639 @@
 # HTTP API

 ## What can I do with this API?
 The API gives another web application control of the pads. The basic functions are

 * create/delete pads
 * grant/forbid access to pads
 * get/set pad content

 The API is designed in a way, so you can reuse your existing user system with their permissions, and map it to Etherpad. Means: Your web application still has to do authentication, but you can tell Etherpad via the api, which visitors should get which permissions. This allows Etherpad to fit into any web application and extend it with real-time functionality. You can embed the pads via an iframe into your website.

 Take a look at [HTTP API client libraries](https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries) to check if a library in your favorite programming language is available.

 ### OpenAPI

 OpenAPI (formerly swagger) definitions are exposed under `/api/openapi.json` (latest) and `/api/{version}/openapi.json`. You can use official tools like [Swagger Editor](https://editor.swagger.io/) to view and explore them.

 ## Examples

 ### Example 1

 A portal (such as WordPress) wants to give a user access to a new pad. Let's assume the user have the internal id 7 and his name is michael.

 Portal maps the internal userid to an etherpad author.

 > Request: `http://pad.domain/api/1/createAuthorIfNotExistsFor?apikey=secret&name=Michael&authorMapper=7`
 >
 > Response: `{code: 0, message:"ok", data: {authorID: "a.s8oes9dhwrvt0zif"}}`

 Portal maps the internal userid to an etherpad group:

 > Request: `http://pad.domain/api/1/createGroupIfNotExistsFor?apikey=secret&groupMapper=7`
 >
 > Response: `{code: 0, message:"ok", data: {groupID: "g.s8oes9dhwrvt0zif"}}`

 Portal creates a pad in the userGroup

 > Request: `http://pad.domain/api/1/createGroupPad?apikey=secret&groupID=g.s8oes9dhwrvt0zif&padName=samplePad&text=This is the first sentence in the pad`
 >
 > Response: `{code: 0, message:"ok", data: null}`

 Portal starts the session for the user on the group:

 > Request: `http://pad.domain/api/1/createSession?apikey=secret&groupID=g.s8oes9dhwrvt0zif&authorID=a.s8oes9dhwrvt0zif&validUntil=1312201246`
 >
 > Response: `{"data":{"sessionID": "s.s8oes9dhwrvt0zif"}}`

 Portal places the cookie "sessionID" with the given value on the client and creates an iframe including the pad.

 ### Example 2

 A portal (such as WordPress) wants to transform the contents of a pad that multiple admins edited into a blog post.

 Portal retrieves the contents of the pad for entry into the db as a blog post:

 > Request: `http://pad.domain/api/1/getText?apikey=secret&padID=g.s8oes9dhwrvt0zif$123`
 >
 > Response: `{code: 0, message:"ok", data: {text:"Welcome Text"}}`

 Portal submits content into new blog post

 > Portal.AddNewBlog(content)
 >

 ## Usage

 ### API version
 The latest version is `1.2.14`

 The current version can be queried via /api.

 ### Request Format

 The API is accessible via HTTP. Starting from **1.8**, API endpoints can be invoked indifferently via GET or POST.

 The URL of the HTTP request is of the form: `/api/ $A P I V E R S I O N /$ FUNCTIONNAME`. $APIVERSION depends on the endpoint you want to use. Depending on the verb you use (GET or POST) **parameters** can be passed differently.

 When invoking via GET (mandatory until **1.7.5** included), parameters must be included in the query string (example: `/api/ $A P I V E R S I O N /$ FUNCTIONNAME?apikey=<APIKEY>&param1=value1`). Please note that starting with nodejs 8.14+ the total size of HTTP request headers has been capped to 8192 bytes. This limits the quantity of data that can be sent in an API request.

 Starting from Etherpad **1.8** it is also possible to invoke the HTTP API via POST. In this case, querystring parameters will still be accepted, but **any parameter with the same name sent via POST will take precedence**. If you need to send large chunks of text (for example, for `setText()`) it is advisable to invoke via POST.

 Example with cURL using GET (toy example, no encoding):
 ```
 curl "http://pad.domain/api/1/setText?apikey=secret&padID=padname&text=this_text_will_NOT_be_encoded_by_curl_use_next_example"
 ```

 Example with cURL using GET (better example, encodes text):
 ```
 curl "http://pad.domain/api/1/setText?apikey=secret&padID=padname" --get --data-urlencode "text=Text sent via GET with proper encoding. For big documents, please use POST"
 ```

 Example with cURL using POST:
 ```
 curl "http://pad.domain/api/1/setText?apikey=secret&padID=padname" --data-urlencode "text=Text sent via POST with proper encoding. For big texts (>8 KB), use this method"
 ```

 ### Response Format
 Responses are valid JSON in the following format:

 ```json
 {
  "code": number,
  "message": string,
  "data": obj
 }
 ```

 * **code** a return code
  * **0** everything ok
  * **1** wrong parameters
  * **2** internal error
  * **3** no such function
  * **4** no or wrong API Key
 * **message** a status message. It's ok if everything is fine, else it contains an error message
 * **data** the payload

 ### Overview

 ![API Overview](https://i.imgur.com/d0nWp.png)

 ## Data Types

 * **groupID**  a string, the unique id of a group. Format is g.16RANDOMCHARS, for example g.s8oes9dhwrvt0zif
 * **sessionID** a string, the unique id of a session. Format is s.16RANDOMCHARS, for example s.s8oes9dhwrvt0zif
 * **authorID** a string, the unique id of an author. Format is a.16RANDOMCHARS, for example a.s8oes9dhwrvt0zif
 * **readOnlyID** a string, the unique id of a readonly relation to a pad. Format is r.16RANDOMCHARS, for example r.s8oes9dhwrvt0zif
 * **padID** a string, format is GROUPID $P A D N A M E, f o r e x a m p l e t h e p a d t e s t o f g r o u p g . s 8 o e s 9 d h w r v t 0 z i f h a s p a d I D g . s 8 o e s 9 d h w r v t 0 z i f$ test

 ### Authentication

 Authentication works via a token that is sent with each request as a post parameter.  There is a single token per Etherpad deployment.  This token will be random string, generated by Etherpad at the first start. It will be saved in APIKEY.txt in the root folder of Etherpad. Only Etherpad and the requesting application knows this key. Token management will not be exposed through this API.

 ### Node Interoperability

 All functions will also be available through a node module accessible from other node.js applications.

 ### JSONP

 The API provides _JSONP_ support to allow requests from a server in a different domain.
 Simply add `&jsonp=?` to the API call.

 Example usage: https://api.jquery.com/jQuery.getJSON/

 ## API Methods

 ### Groups
 Pads can belong to a group. The padID of grouppads is starting with a groupID like g.asdfasdfasdfasdf$test

 #### createGroup()
 * API >= 1

 creates a new group

 *Example returns:*
  * `{code: 0, message:"ok", data: {groupID: g.s8oes9dhwrvt0zif}}`

 #### createGroupIfNotExistsFor(groupMapper)
 * API >= 1

 this functions helps you to map your application group ids to Etherpad group ids

 *Example returns:*
  * `{code: 0, message:"ok", data: {groupID: g.s8oes9dhwrvt0zif}}`

 #### deleteGroup(groupID)
 * API >= 1

 deletes a group

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"groupID does not exist", data: null}`

 #### listPads(groupID)
 * API >= 1

 returns all pads of this group

 *Example returns:*
  * `{code: 0, message:"ok", data: {padIDs : ["g.s8oes9dhwrvt0zif $t e s t ", " g . s 8 o e s 9 d h w r v t 0 z i f$ test2"]}`
  * `{code: 1, message:"groupID does not exist", data: null}`

 #### createGroupPad(groupID, padName [, text])
 * API >= 1

 creates a new pad in this group

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"pad does already exist", data: null}`
  * `{code: 1, message:"groupID does not exist", data: null}`

 #### listAllGroups()
 * API >= 1.1

 lists all existing groups

 *Example returns:*
  * `{code: 0, message:"ok", data: {groupIDs: ["g.mKjkmnAbSMtCt8eL", "g.3ADWx6sbGuAiUmCy"]}}`
  * `{code: 0, message:"ok", data: {groupIDs: []}}`

 ### Author
 These authors are bound to the attributes the users choose (color and name).

 #### createAuthor([name])
 * API >= 1

 creates a new author

 *Example returns:*
  * `{code: 0, message:"ok", data: {authorID: "a.s8oes9dhwrvt0zif"}}`

 #### createAuthorIfNotExistsFor(authorMapper [, name])
 * API >= 1

 this functions helps you to map your application author ids to Etherpad author ids

 *Example returns:*
  * `{code: 0, message:"ok", data: {authorID: "a.s8oes9dhwrvt0zif"}}`

 #### listPadsOfAuthor(authorID)
 * API >= 1

 returns an array of all pads this author contributed to

 *Example returns:*
  * `{code: 0, message:"ok", data: {padIDs: ["g.s8oes9dhwrvt0zif $t e s t ", " g . s 8 o e j k l h w r v t 0 z i f$ foo"]}}`
  * `{code: 1, message:"authorID does not exist", data: null}`

 #### getAuthorName(authorID)
 * API >= 1.1

 Returns the Author Name of the author

 *Example returns:*
  * `{code: 0, message:"ok", data: {authorName: "John McLear"}}`

 -> can't be deleted cause this would involve scanning all the pads where this author was

 ### Session
 Sessions can be created between a group and an author. This allows an author to access more than one group. The sessionID will be set as a cookie to the client and is valid until a certain date. The session cookie can also contain multiple comma-separated sessionIDs, allowing a user to edit pads in different groups at the same time. Only users with a valid session for this group, can access group pads. You can create a session after you authenticated the user at your web application, to give them access to the pads. You should save the sessionID of this session and delete it after the user logged out.

 #### createSession(groupID, authorID, validUntil)
 * API >= 1

 creates a new session. validUntil is an unix timestamp in seconds

 *Example returns:*
  * `{code: 0, message:"ok", data: {sessionID: "s.s8oes9dhwrvt0zif"}}`
  * `{code: 1, message:"groupID doesn't exist", data: null}`
  * `{code: 1, message:"authorID doesn't exist", data: null}`
  * `{code: 1, message:"validUntil is in the past", data: null}`

 #### deleteSession(sessionID)
 * API >= 1

 deletes a session

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"sessionID does not exist", data: null}`

 #### getSessionInfo(sessionID)
 * API >= 1

 returns informations about a session

 *Example returns:*
  * `{code: 0, message:"ok", data: {authorID: "a.s8oes9dhwrvt0zif", groupID: g.s8oes9dhwrvt0zif, validUntil: 1312201246}}`
  * `{code: 1, message:"sessionID does not exist", data: null}`

 #### listSessionsOfGroup(groupID)
 * API >= 1

 returns all sessions of a group

 *Example returns:*
  * `{"code":0,"message":"ok","data":{"s.oxf2ras6lvhv2132":{"groupID":"g.s8oes9dhwrvt0zif","authorID":"a.akf8finncvomlqva","validUntil":2312905480}}}`
  * `{code: 1, message:"groupID does not exist", data: null}`

 #### listSessionsOfAuthor(authorID)
 * API >= 1

 returns all sessions of an author

 *Example returns:*
  * `{"code":0,"message":"ok","data":{"s.oxf2ras6lvhv2132":{"groupID":"g.s8oes9dhwrvt0zif","authorID":"a.akf8finncvomlqva","validUntil":2312905480}}}`
  * `{code: 1, message:"authorID does not exist", data: null}`

 ### Pad Content

 Pad content can be updated and retrieved through the API

 #### getText(padID, [rev])
 * API >= 1

 returns the text of a pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {text:"Welcome Text"}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### setText(padID, text)
 * API >= 1

 Sets the text of a pad.

 If your text is long (>8 KB), please invoke via POST and include `text` parameter in the body of the request, not in the URL (since Etherpad **1.8**).

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`
  * `{code: 1, message:"text too long", data: null}`

 #### appendText(padID, text)
 * API >= 1.2.13

 Appends text to a pad.

 If your text is long (>8 KB), please invoke via POST and include `text` parameter in the body of the request, not in the URL (since Etherpad **1.8**).

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`
  * `{code: 1, message:"text too long", data: null}`

 #### getHTML(padID, [rev])
 * API >= 1

 returns the text of a pad formatted as HTML

 *Example returns:*
  * `{code: 0, message:"ok", data: {html:"Welcome Text<br>More Text"}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### setHTML(padID, html)
 * API >= 1

 sets the text of a pad based on HTML, HTML must be well-formed. Malformed HTML will send a warning to the API log.

 If `html` is long (>8 KB), please invoke via POST and include `html` parameter in the body of the request, not in the URL (since Etherpad **1.8**).

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### getAttributePool(padID)
 * API >= 1.2.8

 returns the attribute pool of a pad

 *Example returns:*
  * `{ "code":0,
       "message":"ok",
       "data": {
         "pool":{
           "numToAttrib":{
             "0":["author","a.X4m8bBWJBZJnWGSh"],
             "1":["author","a.TotfBPzov54ihMdH"],
             "2":["author","a.StiblqrzgeNTbK05"],
             "3":["bold","true"]
           },
           "attribToNum":{
             "author,a.X4m8bBWJBZJnWGSh":0,
             "author,a.TotfBPzov54ihMdH":1,
             "author,a.StiblqrzgeNTbK05":2,
             "bold,true":3
           },
           "nextNum":4
         }
       }
     }`
  * `{"code":1,"message":"padID does not exist","data":null}`

 #### getRevisionChangeset(padID, [rev])
 * API >= 1.2.8

 get the changeset at a given revision, or last revision if 'rev' is not defined.

 *Example returns:*
  * `{ "code" : 0,
       "message" : "ok",
       "data" : "Z:1>6b|5+6b$Welcome to Etherpad!\n\nThis pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!\n\nGet involved with Etherpad at https://etherpad.org\n"
     }`
  * `{"code":1,"message":"padID does not exist","data":null}`
  * `{"code":1,"message":"rev is higher than the head revision of the pad","data":null}`

 #### createDiffHTML(padID, startRev, endRev)
 * API >= 1.2.7

 returns an object of diffs from 2 points in a pad

 *Example returns:*
  * `{"code":0,"message":"ok","data":{"html":"<style>\n.authora_HKIv23mEbachFYfH {background-color: #a979d9}\n.authora_n4gEeMLsv1GivNeh {background-color: #a9b5d9}\n.removed {text-decoration: line-through; -ms-filter:'progid:DXImageTransform.Microsoft.Alpha(Opacity=80)'; filter: alpha(opacity=80); opacity: 0.8; }\n</style>Welcome to Etherpad!<br><br>This pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!<br><br>Get involved with Etherpad at <a href=\"http&#x3a;&#x2F;&#x2F;etherpad&#x2e;org\">http:&#x2F;&#x2F;etherpad.org</a><br><span class=\"authora_HKIv23mEbachFYfH\">aw</span><br><br>","authors":["a.HKIv23mEbachFYfH",""]}}`
  * `{"code":4,"message":"no or wrong API Key","data":null}`

 #### restoreRevision(padId, rev)
 * API >= 1.2.11

 Restores revision from past as new changeset

 *Example returns:*
  * {code:0, message:"ok", data:null}
  * {code: 1, message:"padID does not exist", data: null}

 ### Chat
 #### getChatHistory(padID, [start, end])
 * API >= 1.2.7

 returns

 * a part of the chat history, when `start` and `end` are given
 * the whole chat history, when no extra parameters are given


 *Example returns:*

 * `{"code":0,"message":"ok","data":{"messages":[{"text":"foo","userId":"a.foo","time":1359199533759,"userName":"test"},{"text":"bar","userId":"a.foo","time":1359199534622,"userName":"test"}]}}`
 * `{code: 1, message:"start is higher or equal to the current chatHead", data: null}`
 * `{code: 1, message:"padID does not exist", data: null}`

 #### getChatHead(padID)
 * API >= 1.2.7

 returns the chatHead (last number of the last chat-message) of the pad


 *Example returns:*

 * `{code: 0, message:"ok", data: {chatHead: 42}}`
 * `{code: 1, message:"padID does not exist", data: null}`

 #### appendChatMessage(padID, text, authorID [, time])
 * API >= 1.2.12

 creates a chat message, saves it to the database and sends it to all connected clients of this pad


 *Example returns:*

 * `{code: 0, message:"ok", data: null}`
 * `{code: 1, message:"text is no string", data: null}`

 ### Pad
 Group pads are normal pads, but with the name schema GROUPID $P A D N A M E . A s e c u r i t y m a n a g e r c o n t r o l s a c c e s s o f t h e m a n d i t^{'} s f o r b i d d e n f o r n o r m a l p a d s t o i n c l u d e a$  in the name.

 #### createPad(padID [, text])
 * API >= 1

 creates a new (non-group) pad.  Note that if you need to create a group Pad, you should call **createGroupPad**.
 You get an error message if you use one of the following characters in the padID: "/", "?", "&" or "#".

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does already exist", data: null}`
  * `{code: 1, message:"malformed padID: Remove special characters", data: null}`

 #### getRevisionsCount(padID)
 * API >= 1

 returns the number of revisions of this pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {revisions: 56}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### getSavedRevisionsCount(padID)
 * API >= 1.2.11

 returns the number of saved revisions of this pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {savedRevisions: 42}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### listSavedRevisions(padID)
 * API >= 1.2.11

 returns the list of saved revisions of this pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {savedRevisions: [2, 42, 1337]}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### saveRevision(padID [, rev])
 * API >= 1.2.11

 saves a revision

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### padUsersCount(padID)
 * API >= 1

 returns the number of user that are currently editing this pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {padUsersCount: 5}}`

 #### padUsers(padID)
 * API >= 1.1

 returns the list of users that are currently editing this pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {padUsers: [{colorId:"#c1a9d9","name":"username1","timestamp":1345228793126,"id":"a.n4gEeMLsvg12452n"},{"colorId":"#d9a9cd","name":"Hmmm","timestamp":1345228796042,"id":"a.n4gEeMLsvg12452n"}]}}`
  * `{code: 0, message:"ok", data: {padUsers: []}}`

 #### deletePad(padID)
 * API >= 1

 deletes a pad

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### copyPad(sourceID, destinationID[, force=false])
 * API >= 1.2.8

 copies a pad with full history and chat. If force is true and the destination pad exists, it will be overwritten.

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### copyPadWithoutHistory(sourceID, destinationID[, force=false])
 * API >= 1.2.15

 copies a pad without copying the history and chat. If force is true and the destination pad exists, it will be overwritten.
 Note that all the revisions will be lost! In most of the cases one should use `copyPad` API instead.

 *Example returns:*
 * `{code: 0, message:"ok", data: null}`
 * `{code: 1, message:"padID does not exist", data: null}`

 #### movePad(sourceID, destinationID[, force=false])
 * API >= 1.2.8

 moves a pad. If force is true and the destination pad exists, it will be overwritten.

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### getReadOnlyID(padID)
 * API >= 1

 returns the read only link of a pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {readOnlyID: "r.s8oes9dhwrvt0zif"}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### getPadID(readOnlyID)
 * API >= 1.2.10

 returns the id of a pad which is assigned to the readOnlyID

 *Example returns:*
  * `{code: 0, message:"ok", data: {padID: "p.s8oes9dhwrvt0zif"}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### setPublicStatus(padID, publicStatus)
 * API >= 1

 sets a boolean for the public status of a pad

 *Example returns:*
  * `{code: 0, message:"ok", data: null}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### getPublicStatus(padID)
 * API >= 1

 return true of false

 *Example returns:*
  * `{code: 0, message:"ok", data: {publicStatus: true}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### listAuthorsOfPad(padID)
 * API >= 1

 returns an array of authors who contributed to this pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {authorIDs : ["a.s8oes9dhwrvt0zif", "a.akf8finncvomlqva"]}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### getLastEdited(padID)
 * API >= 1

 returns the timestamp of the last revision of the pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {lastEdited: 1340815946602}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### sendClientsMessage(padID, msg)
 * API >= 1.1

 sends a custom message of type `msg` to the pad

 *Example returns:*
  * `{code: 0, message:"ok", data: {}}`
  * `{code: 1, message:"padID does not exist", data: null}`

 #### checkToken()
 * API >= 1.2

 returns ok when the current api token is valid

 *Example returns:*
  * `{"code":0,"message":"ok","data":null}`
  * `{"code":4,"message":"no or wrong API Key","data":null}`

 ### Pads

 #### listAllPads()
 * API >= 1.2.1

 lists all pads on this epl instance

 *Example returns:*
 * `{code: 0, message:"ok", data: {padIDs: ["testPad", "thePadsOfTheOthers"]}}`

 ### Global

 #### getStats()
 *  API >= 1.2.14

 get stats of the etherpad instance

 *Example returns*
 * `{"code":0,"message":"ok","data":{"totalPads":3,"totalSessions": 2,"totalActivePads": 1}}`

--- a/doc/api/pluginfw.md
+++ b/doc/api/pluginfw.md
@ -0,0 +1,22 @@
 # Plugin Framework

 `require("ep_etherpad-lite/static/js/plugingfw/plugins")`

 ## plugins.update

 `require("ep_etherpad-lite/static/js/plugingfw/plugins").update()` will use npm
 to list all installed modules and read their ep.json files, registering the
 contained hooks. A hook registration is a pair of a hook name and a function
 reference (filename for require() plus function name)

 ## hooks.callAll

 `require("ep_etherpad-lite/static/js/plugingfw/hooks").callAll("hook_name",
 {argname:value})` will call all hook functions registered for `hook_name` with
 `{argname:value}`.

 ## hooks.aCallAll

 ?

 ## ...
--- a/doc/api/toolbar.md
+++ b/doc/api/toolbar.md
@ -0,0 +1,46 @@
 # Toolbar controller
 src/node/utils/toolbar.js

 ## button(opts)
 * {Object} `opts`
   * `command` - this command fill be fired on the editbar on click
   * `localizationId` - will be set as `data-l10-id`
   * `class` - here you can add additional classes to the button

 Returns: {Button}

 Example:
 ```
 var orderedlist = toolbar.button({
  command: "insertorderedlist",
  localizationId: "pad.toolbar.ol.title",
  class: "buttonicon buttonicon-insertorderedlist"
 })
 ```

 You can also create buttons with text:

 ```
 var myButton = toolbar.button({
  command: "myButton",
  localizationId: "myPlugin.toolbar.myButton",
  class: "buttontext"
 })
 ```

 ## selectButton(opts)
 * {Object} `opts`
   * `id` - id of the menu item
   * `selectId` - id of the select element
   * `command` - this command fill be fired on the editbar on change

 Returns: {SelectButton}

 ## SelectButton.addOption(value, text, attributes)
 * {String} value - The value of this option
 * {String} text - the label text used for this option
 * {Object} attributes - any additional html attributes go here (e.g. `data-l10n-id`)

 ## registerButton(name, item)
  * {String} name - used to reference the item in the toolbar config in settings.json
  * {Button|SelectButton} item - the button to add
--- a/doc/assets/style.css
+++ b/doc/assets/style.css
@ -0,0 +1,62 @@
 html {
  border-top: solid green 5pt;
 }

 body.apidoc {
  width: 60%;
  min-width: 10cm;
  margin: 0 auto;
 }

 #header {
  padding: 1pc 0;
  color: #111;
 }

 a,
 a:active {
  color: #272;
 }
 a:focus,
 a:hover {
  color: #050;
 }

 #apicontent a.mark,
 #apicontent a.mark:active {
  float: right;
  color: #BBB;
  font-size: 0.7cm;
  text-decoration: none;
 }
 #apicontent a.mark:focus,
 #apicontent a.mark:hover {
  color: #AAA;
 }

 #apicontent code {
  padding: 1px;
  background-color: #EEE;
  border-radius: 4px;
  border: 1px solid #DDD;
 }
 #apicontent pre>code {
  display: block;
  overflow: auto;
  padding: 5px;
 }

 table, th, td {
  text-align: left;
  border: 1px solid gray;
  border-collapse: collapse;
 }

 th {
  padding: 0.5em;
  background: #EEE;
 }

 td {
  padding: 0.5em;
 }
--- a/doc/cookies.md
+++ b/doc/cookies.md
@ -0,0 +1,18 @@
 # Cookies

 Cookies used by Etherpad.

 | Name            | Sample value                       | Domain      | Path | Expires/max-age | Http-only| Secure |  Usage description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
 |-----------------|------------------------------------|-------------|------|-----------------|----------|--------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 |express_sid      | s%3A7yCNjRmTW8ylGQ53I2IhOwYF9...   | example.org | /    | Session         | true     | true   | Session ID of the [Express web framework](https://expressjs.com). When Etherpad is behind a reverse proxy, and an administrator wants to use session stickiness, he may use this cookie. If you are behind a reverse proxy, please remember to set `trustProxy: true` in `settings.json`. Set in [webaccess.js#L131](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/node/hooks/express/webaccess.js#L131).                                                                                                                                                                                                                                                                                                                                      |
 |language         | en                                 | example.org | /    | Session         | false    | true   | The language of the UI (e.g.: `en-GB`, `it`). Set in [pad_editor.js#L111](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad_editor.js#L111).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
 |prefs / prefsHttp| %7B%22epThemesExtTheme%22...       | example.org | /p   | year 3000       | false    | true   | Client-side preferences (e.g.: font family, chat always visible, show authorship colors, ...). Set in [pad_cookie.js#L49](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad_cookie.js#L49). `prefs` is used if Etherpad is accessed over HTTPS, `prefsHttp` if accessed over HTTP. For more info see https://github.com/ether/etherpad-lite/issues/3179.                                                                                                                                                                                                                                                                                                                                                                             |
 |token            | t.tFzkihhhBf4xKEpCK3PU             | example.org | /    | 60 days         | false    | true   | A random token representing the author, of the form `t.randomstring_of_lenght_20`. The random string is generated by the client, at ([pad.js#L55-L66](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad.js#L55-L66)). This cookie is always set by the client (at [pad.js#L153-L158](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad.js#L153-L158)) without any solicitation from the server. It is used for all the pads accessed via the web UI (not used for the HTTP API). On the server side, its value is accessed at [SecurityManager.js#L33](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/node/db/SecurityManager.js#L33).|

 For more info, visit the related discussion at https://github.com/ether/etherpad-lite/issues/3563.

 Etherpad HTTP API clients may make use (if they choose so) to send another cookie:

 | Name            | Sample value                       | Domain      | Usage description                                                                                                                                                                                                                                                                                                                                                                                                                          |
 |-----------------|------------------------------------|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 |sessionID        | s.1c70968b333b25476a2c7bdd0e0bed17 | example.org | Sessions can be created between a group and an author. This allows an author to access more than one group. The sessionID will be set as a cookie to the client and is valid until a certain date. The session cookie can also contain multiple comma-separated sessionIDs, allowing a user to edit pads in different groups at the same time. More info - https://github.com/ether/etherpad-lite/blob/develop/doc/api/http_api.md#session |
--- a/doc/database.md
+++ b/doc/database.md
@ -0,0 +1,67 @@
 # Database structure

 ## Keys and their values

 ### groups
 A list of all existing groups (a JSON object with groupIDs as keys and `1` as values).

 ### pad:$PADID
 Contains all information about pads

 * **atext** - the latest attributed text
 * **pool** - the attribute pool
 * **head** - the number of the latest revision
 * **chatHead** - the number of the latest chat entry
 * **public** - flag that disables security for this pad
 * **passwordHash** - string that contains a salted sha512 sum of this pad's password

 ### pad: $P A D I D : r e v s :$ REVNUM
 Saves a revision  $R E V N U M o f p a d$ PADID

 * **meta**
  * **author** - the autorID of this revision
  * **timestamp** - the timestamp of when this revision was created
 * **changeset** - the changeset of this revision

 ### pad: $P A D I D : c h a t :$ CHATNUM
 Saves a chat entry with num  $C H A T N U M o f p a d$ PADID

 * **text** - the text of this chat entry
 * **userId** - the authorID of this chat entry
 * **time** - the timestamp of this chat entry

 ### pad2readonly:$PADID
 Translates a padID to a readonlyID
 ### readonly2pad:$READONLYID
 Translates a readonlyID to a padID
 ### token2author:$TOKENID
 Translates a token to an authorID
 ### globalAuthor:$AUTHORID
 Information about an author

 * **name** - the name of this author as shown in the pad
 * **colorID** - the colorID of this author as shown in the pad

 ### mapper2group:$MAPPER
 Maps an external application identifier to an internal group
 ### mapper2author:$MAPPER
 Maps an external application identifier to an internal author
 ### group:$GROUPID
 a group of pads

 * **pads** - object with pad names in it, values are 1
 ### session:$SESSIONID
 a session between an author and a group

 * **groupID** - the groupID the session belongs too
 * **authorID** - the authorID the session belongs too
 * **validUntil** - the timestamp until this session is valid

 ### author2sessions:$AUTHORID
 saves the sessions of an author

 * **sessionsIDs** - object with sessionIDs in it, values are 1

 ### group2sessions:$GROUPID

 * **sessionsIDs** - object with sessionIDs in it, values are 1
--- a/doc/docker.md
+++ b/doc/docker.md
@ -0,0 +1,226 @@
 # Docker

 The official Docker image is available on https://hub.docker.com/r/etherpad/etherpad.

 ## Downloading from Docker Hub
 If you are ok downloading a [prebuilt image from Docker Hub](https://hub.docker.com/r/etherpad/etherpad), these are the commands:
 ```bash
 # gets the latest published version
 docker pull etherpad/etherpad

 # gets a specific version
 docker pull etherpad/etherpad:1.8.0
 ```

 ## Build a personalized container

 If you want to use a personalized settings file, **you will have to rebuild your image**.
 All of the following instructions are as a member of the `docker` group.
 By default, the Etherpad Docker image is built and run in `production` mode: no development dependencies are installed, and asset bundling speeds up page load time.

 ### Rebuilding with custom settings
 Edit `<BASEDIR>/settings.json.docker` at your will. When rebuilding the image, this file will be copied inside your image and renamed to `setting.json`.

 **Each configuration parameter can also be set via an environment variable**, using the syntax `"${ENV_VAR}"` or `"${ENV_VAR:default_value}"`. For details, refer to `settings.json.template`.

 ### Rebuilding including some plugins
 If you want to install some plugins in your container, it is sufficient to list them in the ETHERPAD_PLUGINS build variable.
 The variable value has to be a space separated, double quoted list of plugin names (see examples).

 Some plugins will need personalized settings. Just refer to the previous section, and include them in your custom `settings.json.docker`.

 ### Examples

 Build a Docker image from the currently checked-out code:
 ```bash
 docker build --tag <YOUR_USERNAME>/etherpad .
 ```

 Include two plugins in the container:
 ```bash
 docker build --build-arg ETHERPAD_PLUGINS="ep_comments_page ep_author_neat" --tag <YOUR_USERNAME>/etherpad .
 ```

 ## Running your instance:

 To run your instance:
 ```bash
 docker run --detach --publish <DESIRED_PORT>:9001 <YOUR_USERNAME>/etherpad
 ```

 And point your browser to `http://<YOUR_IP>:<DESIRED_PORT>`

 ## Options available by default

 The `settings.json.docker` available by default allows to control almost every setting via environment variables.

 ### General

 | Variable           | Description                                                                                | Default                                                                                                                                                                                                                             |
 | ------------------ | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `TITLE`            | The name of the instance                                                                   | `Etherpad`                                                                                                                                                                                                                          |
 | `FAVICON`          | favicon default name, or a fully specified URL to your own favicon                         | `favicon.ico`                                                                                                                                                                                                                       |
 | `DEFAULT_PAD_TEXT` | The default text of a pad                                                                  | `Welcome to Etherpad! This pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents! Get involved with Etherpad at https://etherpad.org` |
 | `IP`               | IP which etherpad should bind at. Change to `::` for IPv6                                  | `0.0.0.0`                                                                                                                                                                                                                           |
 | `PORT`             | port which etherpad should bind at                                                         | `9001`                                                                                                                                                                                                                              |
 | `ADMIN_PASSWORD`   | the password for the `admin` user (leave unspecified if you do not want to create it)      |                                                                                                                                                                                                                                     |
 | `USER_PASSWORD`    | the password for the first user `user` (leave unspecified if you do not want to create it) |                                                                                                                                                                                                                                     |


 ### Database

 | Variable      | Description                                                    | Default                                                               |
 | ------------- | -------------------------------------------------------------- | --------------------------------------------------------------------- |
 | `DB_TYPE`     | a database supported by https://www.npmjs.com/package/ueberdb2 | not set, thus will fall back to `DirtyDB` (please choose one instead) |
 | `DB_HOST`     | the host of the database                                       |                                                                       |
 | `DB_PORT`     | the port of the database                                       |                                                                       |
 | `DB_NAME`     | the database name                                              |                                                                       |
 | `DB_USER`     | a database user with sufficient permissions to create tables   |                                                                       |
 | `DB_PASS`     | the password for the database username                         |                                                                       |
 | `DB_CHARSET`  | the character set for the tables (only required for MySQL)     |                                                                       |
 | `DB_FILENAME` | in case `DB_TYPE` is `DirtyDB`, the database filename.         | `var/dirty.db`                                                        |

 If your database needs additional settings, you will have to use a personalized `settings.json.docker` and rebuild the container (or otherwise put the updated `settings.json` inside your image).


 ### Pad Options

 | Variable                         | Description | Default |
 | -------------------------------- | ----------- | ------- |
 | `PAD_OPTIONS_NO_COLORS`          |             | `false` |
 | `PAD_OPTIONS_SHOW_CONTROLS`      |             | `true`  |
 | `PAD_OPTIONS_SHOW_CHAT`          |             | `true`  |
 | `PAD_OPTIONS_SHOW_LINE_NUMBERS`  |             | `true`  |
 | `PAD_OPTIONS_USE_MONOSPACE_FONT` |             | `false` |
 | `PAD_OPTIONS_USER_NAME`          |             | `false` |
 | `PAD_OPTIONS_USER_COLOR`         |             | `false` |
 | `PAD_OPTIONS_RTL`                |             | `false` |
 | `PAD_OPTIONS_ALWAYS_SHOW_CHAT`   |             | `false` |
 | `PAD_OPTIONS_CHAT_AND_USERS`     |             | `false` |
 | `PAD_OPTIONS_LANG`               |             | `en-gb` |


 ### Shortcuts

 | Variable                            | Description                                      | Default |
 | ----------------------------------- | ------------------------------------------------ | ------- |
 | `PAD_SHORTCUTS_ENABLED_ALT_F9`      | focus on the File Menu and/or editbar            | `true`  |
 | `PAD_SHORTCUTS_ENABLED_ALT_C`       | focus on the Chat window                         | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_S`       | save a revision                                  | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_Z`       | undo/redo                                        | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_Y`       | redo                                             | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_I`       | italic                                           | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_B`       | bold                                             | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_U`       | underline                                        | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_H`       | backspace                                        | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_5`       | strike through                                   | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_1` | ordered list                                     | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_2` | shows a gritter popup showing a line author      | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_L` | unordered list                                   | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_N` | ordered list                                     | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_C` | clear authorship                                 | `true`  |
 | `PAD_SHORTCUTS_ENABLED_DELETE`      |                                                  | `true`  |
 | `PAD_SHORTCUTS_ENABLED_RETURN`      |                                                  | `true`  |
 | `PAD_SHORTCUTS_ENABLED_ESC`         | in mozilla versions 14-19 avoid reconnecting pad | `true`  |
 | `PAD_SHORTCUTS_ENABLED_TAB`         | indent                                           | `true`  |
 | `PAD_SHORTCUTS_ENABLED_CTRL_HOME`   | scroll to top of pad                             | `true`  |
 | `PAD_SHORTCUTS_ENABLED_PAGE_UP`     |                                                  | `true`  |
 | `PAD_SHORTCUTS_ENABLED_PAGE_DOWN`   |                                                  | `true`  |


 ### Skins

 You can use the UI skin variants builder at `/p/test#skinvariantsbuilder`

 For the colibris skin only, you can choose how to render the three main containers:
  * toolbar (top menu with icons)
  * editor (containing the text of the pad)
  * background (area outside of editor, mostly visible when using page style)

 For each of the 3 containers you can choose 4 color combinations:
   * super-light
   * light
   * dark
   * super-dark

 For the editor container, you can also make it full width by adding `full-width-editor` variant (by default editor is rendered as a page, with a max-width of 900px).

 | Variable        | Description                                                                    | Default                                                   |
 | --------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------- |
 | `SKIN_NAME`     | either `no-skin`, `colibris` or an existing directory under `src/static/skins` | `colibris`                                                |
 | `SKIN_VARIANTS` | multiple skin variants separated by spaces                                     | `super-light-toolbar super-light-editor light-background` |


 ### Logging

 | Variable             | Description                                          | Default |
 | -------------------- | ---------------------------------------------------- | ------- |
 | `LOGLEVEL`           | valid values are `DEBUG`, `INFO`, `WARN` and `ERROR` | `INFO`  |
 | `DISABLE_IP_LOGGING` | Privacy: disable IP logging                          | `false` |


 ### Advanced

 | Variable                          | Description                                                                                                                                                                                            | Default            |
 | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------ |
 | `SHOW_SETTINGS_IN_ADMIN_PAGE`     | hide/show the settings.json in admin page                                                                                                                                                              | `true`             |
 | `TRUST_PROXY`                     | set to `true` if you are using a reverse proxy in front of Etherpad (for example: Traefik for SSL termination via Let's Encrypt). This will affect security and correctness of the logs if not done    | `false`            |
 | `IMPORT_MAX_FILE_SIZE`            | maximum allowed file size when importing a pad, in bytes.                                                                                                                                              | `52428800` (50 MB) |
 | `IMPORT_EXPORT_MAX_REQ_PER_IP`    | maximum number of import/export calls per IP.                                                                                                                                                          | `10`               |
 | `IMPORT_EXPORT_RATE_LIMIT_WINDOW` | the call rate for import/export requests will be estimated in this time window (in milliseconds)                                                                                                       | `90000`            |
 | `SUPPRESS_ERRORS_IN_PAD_TEXT`     | Should we suppress errors from being visible in the default Pad Text?                                                                                                                                  | `false`            |
 | `REQUIRE_SESSION`                 | If this option is enabled, a user must have a session to access pads. This effectively allows only group pads to be accessed.                                                                          | `false`            |
 | `EDIT_ONLY`                       | Users may edit pads but not create new ones. Pad creation is only via the API. This applies both to group pads and regular pads.                                                                       | `false`            |
 | `MINIFY`                          | If true, all css & js will be minified before sending to the client. This will improve the loading performance massively, but makes it difficult to debug the javascript/css                           | `true`             |
 | `MAX_AGE`                         | How long may clients use served javascript code (in seconds)? Not setting this may cause problems during deployment. Set to 0 to disable caching.                                                      | `21600` (6 hours)  |
 | `ABIWORD`                         | Absolute path to the Abiword executable. Abiword is needed to get advanced import/export features of pads. Setting it to null disables Abiword and will only allow plain text and HTML import/exports. | `null`             |
 | `SOFFICE`                         | This is the absolute path to the soffice executable. LibreOffice can be used in lieu of Abiword to export pads. Setting it to null disables LibreOffice exporting.                                     | `null`             |
 | `TIDY_HTML`                       | Path to the Tidy executable. Tidy is used to improve the quality of exported pads. Setting it to null disables Tidy.                                                                                   | `null`             |
 | `ALLOW_UNKNOWN_FILE_ENDS`         | Allow import of file types other than the supported ones: txt, doc, docx, rtf, odt, html & htm                                                                                                         | `true`             |
 | `REQUIRE_AUTHENTICATION`          | This setting is used if you require authentication of all users. Note: "/admin" always requires authentication.                                                                                        | `false`            |
 | `REQUIRE_AUTHORIZATION`           | Require authorization by a module, or a user with is_admin set, see below.                                                                                                                             | `false`            |
 | `AUTOMATIC_RECONNECTION_TIMEOUT`  | Time (in seconds) to automatically reconnect pad when a "Force reconnect" message is shown to user. Set to 0 to disable automatic reconnection.                                                        | `0`                |
 | `FOCUS_LINE_PERCENTAGE_ABOVE`     | Percentage of viewport height to be additionally scrolled. e.g. 0.5, to place caret line in the middle of viewport, when user edits a line above of the viewport. Set to 0 to disable extra scrolling  | `0`                |
 | `FOCUS_LINE_PERCENTAGE_BELOW`     | Percentage of viewport height to be additionally scrolled. e.g. 0.5, to place caret line in the middle of viewport, when user edits a line below of the viewport. Set to 0 to disable extra scrolling  | `0`                |
 | `FOCUS_LINE_PERCENTAGE_ARROW_UP`  | Percentage of viewport height to be additionally scrolled when user presses arrow up in the line of the top of the viewport. Set to 0 to let the scroll to be handled as default by Etherpad           | `0`                |
 | `FOCUS_LINE_DURATION`             | Time (in milliseconds) used to animate the scroll transition. Set to 0 to disable animation                                                                                                            | `0`                |
 | `FOCUS_LINE_CARET_SCROLL`         | Flag to control if it should scroll when user places the caret in the last line of the viewport                                                                                                        | `false`            |
 | `LOAD_TEST`                       | Allow Load Testing tools to hit the Etherpad Instance. WARNING: this will disable security on the instance.                                                                                            | `false`            |
 | `EXPOSE_VERSION`                  | Expose Etherpad version in the web interface and in the Server http header. Do not enable on production machines.                                                                                      | `false`            |


 ### Examples

 Use a Postgres database, no admin user enabled:

 ```shell
 docker run -d \
 	--name etherpad         \
 	-p 9001:9001            \
 	-e 'DB_TYPE=postgres'   \
 	-e 'DB_HOST=db.local'   \
 	-e 'DB_PORT=4321'       \
 	-e 'DB_NAME=etherpad'   \
 	-e 'DB_USER=dbusername' \
 	-e 'DB_PASS=mypassword' \
 	etherpad/etherpad
 ```

 Run enabling the administrative user `admin`:

 ```shell
 docker run -d \
 	--name etherpad \
 	-p 9001:9001 \
 	-e 'ADMIN_PASSWORD=supersecret' \
 	etherpad/etherpad
 ```

 Run a test instance running DirtyDB on a persistent volume:

 ```
 docker run -d \
 	-v etherpad_data:/opt/etherpad-lite/var \
 	-p 9001:9001 \
 	etherpad/etherpad
 ```
--- a/doc/documentation.md
+++ b/doc/documentation.md
@ -0,0 +1,15 @@
 # About this Documentation

 <!-- type=misc -->

 The goal of this documentation is to comprehensively explain Etherpad,
 both from a reference as well as a conceptual point of view.

 Where appropriate, property types, method arguments, and the arguments
 provided to event handlers are detailed in a list underneath the topic
 heading.

 Every `.html` file is generated based on the corresponding
 `.md` file in the `doc/api/` folder in the source tree. The
 documentation is generated using the `bin/doc/generate.js` program.
 The HTML template is located at `doc/template.html`.
--- a/doc/easysync/README.md
+++ b/doc/easysync/README.md
@ -0,0 +1,2 @@
 # About this folder
 We put all documentations we found about the original Etherpad together in this folder. Most of this is still valid for the current (node.js based) Etherpad.
--- a/doc/easysync/easysync-full-description.pdf
+++ b/doc/easysync/easysync-full-description.pdf
--- a/doc/easysync/easysync-full-description.tex
+++ b/doc/easysync/easysync-full-description.tex
@ -0,0 +1,372 @@
 \documentclass{article} 
 \usepackage{hyperref}

 \begin{document}

 \title{Etherpad and EasySync Technical Manual}
 \author{AppJet, Inc., with modifications by the Etherpad Foundation}
 \date{\today}

 \maketitle

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 \tableofcontents
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

 \section{Documents}
 \begin{itemize}
 \item A document is a list of characters, or a string.
 \item A document can also be represented as a list of \emph{changesets}.
 \end{itemize}

 \section{Changesets}

 \begin{itemize}
 \item A changeset represents a change to a document.
 \item A changeset can be applied to a document to produce a new document.
 \item When a document is represented as a list of changesets, it is assumed that the first changeset applies to the empty document, [].
 \end{itemize}


 \section{Changeset representation} \label{representation}

 $$(\ell \rightarrow \ell')[c_1,c_2,c_3,...]$$

 where

 \begin{itemize}
 \item[] $\ell$ is the length of the document before the change,
 \item[] $\ell'$ is the length of the document after the change,
 \item[] $[c_1,c_2,c_3,...]$ is an array of $\ell'$ characters that described the document after the change.
 \end{itemize}

 Note that $\forall c_i : 0 \leq i \leq \ell'$ is either an integer or a character.

 \begin{itemize}
 \item Integers represent retained characters in the original document.
 \item Characters represent insertions.
 \end{itemize}

 \section{Constraints on Changesets}

 \begin{itemize}
 \item Changesets are canonical and therefor comparable.  When represented in computer memory, we always use the same representation for the same changeset.  If the memory representation of two changesets differ, they must be different changesets.
 \item Changesets are compact.  Thus, if there are two ways to represent a changeset in computer memory, then we always use the representation that takes up the fewest bytes.
 \end{itemize}

 Later we will discuss optimizations to changeset
 representation (using ``strips'' and other such
 techniques).  The two constraints must apply to any
 representation of changesets.

 \section{Notation}

 \begin{itemize}
 \item We use the algebraic multiplication notation to represent changeset application.
 \item While changesets are defined as operations on documents, documents themselves are represented as a list of changesets, initially applying to the empty document.
 \end{itemize}

 \paragraph{Example}
 $A=(0\rightarrow 5)[``hello"]$
 $B=(5\rightarrow 11)[0-4, ``\ world"]$

 We can write the document ``hello world'' as $A\cdot B$ or
 just $AB$.  Note that the ``initial document'' can be made
 into the changeset $(0\rightarrow
 N)[``<\mathit{the\ document\ text}>"]$.

 When $A$ and $B$ are changesets, we can also refer to $(AB)$ as ``the composition'' of $A$ and $B$.  Changesets are closed under composition.

 \section{Composition of Changesets}

 For any two changesets $A$, $B$ such that

 \begin{itemize}
 \item[] $A=(n_1\rightarrow n_2)[\cdots]$
 \item[] $B=(n_2\rightarrow n_3)[\cdots]$
 \end{itemize}
 it is clear that there is a third changeset $C=(n_1\rightarrow n_3)[\cdots]$ such that applying $C$ to a document $X$ yields the same resulting document as does applying $A$ and then $B$.  In this case, we write $AB=C$.

 Given the representation from Section \ref{representation}, it is straightforward to compute the composition of two changesets.

 \section{Changeset Merging}

 Now we come to realtime document editing.  Suppose two different users make two different changes to the same document at the same time.  It is impossible to compose these changes.    For example, if we have the document $X$ of length $n$, we may have $A=(n\rightarrow n_a)[\ldots n_a \mathrm{characters}]$,  $B=(n\rightarrow n_b)[\ldots n_b \mathrm{characters}]$ where $n\neq n_a\neq n_b$.

 It is impossible to compute $(XA)B$ because $B$ can only be applied to a document of length $n$, and $(XA)$ has length $n_a$.  Similarly, $A$ cannot be applied to $(XB)$ because $(XB)$ has length $n_b$.

 This is where \emph{merging} comes in.  Merging takes two changesets that apply to the same initial document (and that cannot be composed), and computes a single new changeset that preserves the intent of both changes.  The merge of $A$ and $B$ is written as $m(A,B)$.  For the Etherpad system to work, we require that $m(A,B)=m(B,A)$.

 Aside from what we have said so far about merging, there are many different implementations that will lead to a workable system.  We have created one implementation for text that has the following constraints.

 \section{Follows} \label{follows}

 When users $A$ and $B$ have the same document $X$ on their screen, and they proceed to make respective changesets $A$ and $B$, it is no use to compute $m(A,B)$, because $m(A,B)$ applies to document $X$, but the users are already looking at document $XA$ and $XB$.  What we really want is to compute $B'$ and $A'$ such that
 $$XAB' = XBA' = Xm(A,B)$$

 ``Following'' computes these $B'$ and $A'$ changesets.  The definition of the ``follow'' function $f$ is such that $Af(A,B)=Bf(B,A)=m(A,B)=m(B,A)$.  When we compute $f(A,B)$
 \begin{itemize}
 \item Insertions in $A$ become retained characters in $f(A,B)$
 \item Insertions in $B$ become insertions in $f(A,B)$
 \item Retain whatever characters are retained in \emph{both} $A$ and $B$
 \end{itemize}

 \paragraph{Example}

 Suppose we have the initial document $X=(0\rightarrow 8)[``\mathit{baseball}"]$ and user $A$ changes it to ``basil'' with changeset $A$, and user $B$ changes it to ``below'' with changeset $B$.

 We have
 $X=(0\rightarrow 8)[``\mathit{baseball}"]$ \
 $A=(8\rightarrow 5)[0-1, ``\mathit{si}", 7]$ \
 $B=(8\rightarrow 5)[0, ``\mathit{e}", 6, ``\mathit{ow}"]$ \

 First we compute the merge $m(A,B)=m(B,A)$ according to the constraints

 $$m(A,B)=(8\rightarrow 6)[0, "e", "si", "ow"] = (8\rightarrow 6)[0, ``\mathit{esiow}"]$$

 Then we need to compute the follows $B'=f(A,B)$ and $A'=f(B,A)$.

 $$B'=f(A,B)=(5\rightarrow 6)[0,``\mathit{e}",2,3,``\mathit{ow}"]$$

 Note that the numbers $0$, $2$, and $3$ are indices into $A=(8\rightarrow 5)[0,1,``\mathit{si}",7]$

 \begin{tabular}{ccccc}
 0 & 1 & 2 & 3 & 4 \
 0 & 1 & s & i & 7
 \end{tabular}

 $A'=f(B,A)=(5\rightarrow 6)[0,1,"si",3,4]$

 We can now double check that $AB'=BA'=m(A,B)=(8\rightarrow 6)[0,``\mathit{esiow}"]$.

 Now that we have made the mathematical meaning of the
 preceding pages complete, we can build a client/server
 system to support realtime editing by multiple users.

 \section{System Overview}

 There is a server that holds the current state of a
 document.  Clients (users) can connect to the server from
 their web browsers.  The clients and server maintain state
 and can send messages to one another in real-time, but
 because we are in a web browser scenario, clients cannot
 send each other messages directly, and must go through the
 server always.  (This may distinguish from prior art?)

 The other critical design feature of the system is that
 \emph{A client must always be able to edit their local
  copy of the document, so the user is never blocked from
  typing because of waiting to send or receive data.}

 \section{Client State}

 At any moment in time, a client maintains its state in the
 form of 3 changesets.  The client document looks like
 $A\cdot X \cdot Y$, where

 $A$ is the latest server version, the composition of all
 changesets committed to the server, from this client or
 from others, that the server has informed this client
 about.  Initially $A=(0\rightarrow N)[<\mathit{initial\ document\ text}>]$.

 $X$ is the composition of all changesets this client has
 submitted to the server but has not heard back about yet.
 Initially $X=(N\rightarrow N)[0,1,2,\ldots, N-1]$, in
 other words, the identity, henceforth denoted $I_N$.

 $Y$ is the composition of all changesets this client has
 made but has not yet submitted to the server yet.
 Initially $Y=(N\rightarrow N)[0,1,2,\ldots, N-1]$.

 \section{Client Operations}

 A client can do 5 things.

 \begin{enumerate}
 \item Incorporate new typing into local state
 \item Submit a changeset to the server
 \item Hear back acknowledgement of a submitted changeset
 \item Hear from the server about other clients' changesets
 \item Connect to the server and request the initial document
 \end{enumerate}

 As these 5 events happen, the client updates its
 representation $A\cdot X \cdot Y$ according to the
 relations that follow.  Changes ``move left'' as time goes
 by: into $Y$ when the user types, into $X$ when change
 sets are submitted to the server, and into $A$ when the
 server acknowledges changesets.

 \subsection{New local typing}

 When a user makes an edit $E$ to the document, the client
 computes the composition $(Y\cdot E)$ and updates its local
 state, i.e. $Y \leftarrow Y\cdot E$.  I.e., if $Y$ is the
 variable holding local unsubmitted changes, it will be
 assigned the new value $(Y\cdot E)$.

 \subsection{Submitting changesets to server}

 When a client submit its local changes to the server, it
 transmits a copy of $Y$ and then assigns $Y$ to $X$, and
 assigns the identity to $Y$.  I.e.,

 \begin{enumerate}
 \item Send $Y$ to server,
 \item $X \leftarrow Y$
 \item $Y \leftarrow I_N$
  (the identity).
 \end{enumerate}

 This happens every 500ms as long as it receives an
 acknowledgement.  Must receive ACK before submitting
 again.  Note that $X$ is always equal to the identity
 before the second step occurs, so no information is lost.

 \subsection{Hear ACK from server}

 When the client hears ACK from server,

 $A \leftarrow A\cdot X$ \
 $X \leftarrow I_N$

 \subsection{Hear about another client's changeset}

 When a client hears about another client's changeset $B$,
 it computes a new $A$, $X$, and $Y$, which we will call
 $A'$, $X'$, and $Y'$ respectively.  It also computes a
 changeset $D$ which is applied to the current text view on
 the client, $V$.  Because $AXY$ must always equal the
 current view, $AXY=V$ before the client hears about $B$,
 and $A'X'Y'=VD$ after the computation is performed.

 The steps are:

 \begin{enumerate}
 \item Compute $A' = AB$
 \item Compute $X' = f(B,X)$
 \item Compute $Y' = f(f(X,B), Y)$
 \item Compute $D=f(Y,f(X,B))$
 \item Assign $A \leftarrow A'$, $X \leftarrow X'$, $Y \leftarrow Y'$.
 \item Apply $D$ to the current view of the document
  displayed on the user's screen.
 \end{enumerate}

 In steps 2,3, and 4, $f$ is the follow operation described
 in Section \ref{follows}.

 \paragraph{Proof that $\mathbf{AXY=V \Rightarrow A'X'Y'=VD}$.}
 Substituting $A'X'Y'=(AB)(f(B,X))(f(f(X,B),Y))$, we
 recall that merges are commutative.  So for any two
 changesets $P$ and $Q$, 
 $$m(P,Q)=m(Q,P)=Qf(Q,P)=Pf(P,Q)$$

 Applying this to the relation above, we see
 \begin{eqnarray*}
 A'X'Y'&=& AB f(B,X) f(f(X,B),Y) \
      &=&AX f(X,B) f(f(X,B),Y) \
      &=&A X Y f(Y, f(X,B)) \
      &=&A X Y D \
      &=&V D 
 \end{eqnarray*}
 As claimed.

 \subsection{Connect to server}

 When a client connects to the server for the first time,
 it first generates a random unique ID and sends this to
 the server.  The client remembers this ID and sends it
 with each changeset to the server.

 The client receives the latest version of the document
 from the server, called HEADTEXT.  The client then sets

 \begin{itemize}
 \item[] $A \leftarrow \mathrm{HEADTEXT}$
 \item[] $X \leftarrow I_N$
 \item[] $Y \leftarrow I_N$
 \end{itemize}

 And finally, the client displays HEADTEXT on the screen.

 \section{Server Overview}

 Like the client(s), the server has state and performs
 operations.  Operations are only performed in response to
 messages from clients.

 \section{Server State}

 The server maintains a document as an ordered list of
 \emph{revision records}.  A revision record is a data
 structure that contains a changeset and authorship
 information.

 \begin{verbatim}
 RevisionRecord = {
  ChangeSet,
  Source (unique ID),
  Revision Number (consecutive order, starting at 0)
 }
 \end{verbatim}

 For efficiency, the server may also store a variable
 called HEADTEXT, which is the composition of all
 changesets in the list of revision records.  This is an
 optimization, because clearly this can be computed from
 the set of revision records.

 \section{Server Operations Overview}

 The server does two things in addition to maintaining
 state representing the set of connected clients and
 remembering what revision number each client is up to date
 with:

 \begin{enumerate}
 \item Respond to a client's connection requesting the initial document.
 \item Respond to a client's submission of a new changeset.
 \end{enumerate}

 \subsection{Respond to client connect}
 When a server receives a connection request from a client,
 it receives the client's unique ID and stores that in the
 server's set of connected clients.  It then sends the
 client the contents of HEADTEXT, and the corresponding
 revision number.  Finally the server notes that this
 client is up to date with that revision number.

 \subsection{Respond to client changeset}

 When the server receives information from a client about
 the client's changeset $C$, it does five things:

 \begin{enumerate}
 \item Notes that this change applies to revision number
  $r_c$ (the client's latest revision).
 \item Creates a new changeset $C'$ that is relative to the
  server's most recent revision number, which we call
  $r_H$ ($H$ for HEAD).  $C'$ can be computed using
  follows (Section \ref{follows}).  Remember that the server has a series of
  changesets, 
 $$S_0\rightarrow S_1\rightarrow \ldots S_{r_c}\rightarrow S_{r_c+1} \rightarrow \ldots \rightarrow S_{r_H} $$
 $C$ is relative to $S_{r_c}$, but we need to compute $C'$ relative to $S_{r_H}$.
 We can compute a new $C$ relative to $S_{r_c+1}$ by computing $f(S_{r_c+1},C)$.  Similarly we can repeat for
 $S_{r_c+2}$ and so forth until we have $C'$ represented relative to $S_{r_H}$.
 \item Send $C'$ to all other clients
 \item Send ACK back to original client
 \item Add $C'$ to the server's list of revision records by creating a new revision record out of this and the client's ID.

 \appendix

 \section*{Additional topics}
 \begin{enumerate}
 \item Optimizations (strips, more caching, etc.)
 \item Pseudocode for composition, merge, and follow
 \item How authorship information is used to color-code the document based on who typed what
 \item How persistent connections are maintained between client and server
 \end{enumerate}
 \end{enumerate}


 \end{document}
--- a/doc/easysync/easysync-notes.pdf
+++ b/doc/easysync/easysync-notes.pdf
--- a/doc/easysync/easysync-notes.tex
+++ b/doc/easysync/easysync-notes.tex
@ -0,0 +1,200 @@
 \documentclass[12pt]{article}

 \usepackage[T1]{fontenc}
 \usepackage[USenglish]{babel} 


 \begin{document}

 \title{Easysync Protocol}
 \author{AppJet, Inc., with modifications by the Etherpad Foundation}
 \date{\today}

 \maketitle

 \section{Attributes}

 An ``attribute'' is a (key,value) pair such as
 \verb|(author,abc123)| or \verb|(bold,true)|.
 Sometimes an attribute is treated as an instruction to add
 that attribute, in which case an empty value means to
 remove it.  So \verb|(bold,)| removes the ``bold''
 attribute.  Attributes are interned and given numeric IDs,
 so the number ``\verb|6|'' could represent
 ``\verb|(bold,true)|'', for example.  This mapping is
 stored in an attribute pool which may be shared by
 multiple changesets.

 Entries in the pool must be unique, so that attributes can
 be compared by their IDs.  Attribute names cannot contain
 commas.

 A changeset looks something like the following:

 \begin{verbatim}
 Z:5g>1|5=2p=v*4*5+1$x
 \end{verbatim}

 With the corresponding pool containing these entries (among others):

 \begin{itemize}
 \item[] \verb|4| $\rightarrow$ \verb|(author,1059348573)|
 \item[] \verb|5| $\rightarrow$ \verb|(bold,true)|
 \end{itemize}

 This changeset, together with the attribute pool,
 represents inserting a bold letter ``x'' into the middle
 of a line.

 The string consists of:

 \begin{itemize}
 \item a letter \verb|Z| (the ``magic character'' and
  format version identifier)
 \item a series punctuation marks (operation codes or
  ``opcodes'' for short), together with alphanumerics
  (numeric values in base 36).
 \item a dollar sign (\verb|$|)
 \item a string of characters used for insertion operations
  (the ``char bank'')
 \end{itemize}

 In the example above, if we separate out the operations
 and convert the numbers to base 10, then we get:
 \begin{verbatim}
 Z :196 >1 |5=97 =31 *4 *5 +1 $x
 \end{verbatim}
 Here are descriptions of the operations, where capital
 letters are variables:

 \begin{description}
 \item{{\bf :N}} \quad \ 
 Source text has length $N$ (must be first op)
 \item{{\bf >N}} \quad \
 Final text is $N$ (positive) characters longer than source
 text (must be second op)
 \item{{\bf <N }} \quad \
 Final text is $N$ (positive) characters shorter than
 source text (must be second op)
 \item{{\bf >0 }} \quad \
 Final text is same length as source text
 \item{{\bf +N }} \quad \
 Insert $N$ characters from the bank, none of them newlines
 \item{{\bf -N}} \quad \
 Skip over (delete) $N$ characters from the source text,
 none of them newlines
 \item{{\bf =N}} \quad \
 Keep $N$ characters from the source text, none of them newlines
 \item{{\bf |L+N}} \quad \
 Insert $N$ characters from the source text, containing $L$
 newlines.  The last character inserted MUST be a newline,
 but not the (new) document's final newline.
 \item{{\bf |L-N}}  \quad \
 Delete $N$ characters from the source text, containing $L$
 newlines. The last character inserted MUST be a newline,
 but not the (old) document's final newline.
 \item{{\bf |L=N}}  \quad \
 Keep $N$ characters from the source text, containing L
 newlines.  The last character kept MUST be a newline, and
 the final newline of the document is allowed.
 \item{{\bf *I}} \quad \
 Apply attribute $I$ from the pool to the following
 \verb|+|, \verb|=|, \verb_|+_, or \verb_|=_ command. In
 other words, any number of \verb|*| ops can come before a
 \verb_+_, \verb_=_, or \verb_|_ but not between a \verb_|_
 and the corresponding \verb_+_ or \verb_=_. If \verb_+_,
 text is inserted having this attribute.  If \verb_=_, text
 is kept but with the attribute applied as an attribute
 addition or removal. Consecutive attributes must be sorted
 lexically by (key,value) with key and value taken as
 strings.  It's illegal to have duplicate keys for
 (key,value) pairs that apply to the same text.  It's
 illegal to have an empty value for a key in the case of an
 insertion (\verb_+_), the pair should just be omitted.
 \end{description}

 Characters from the source text that aren't accounted for
 are assumed to be kept with the same attributes.

 \paragraph{Additional Constraints}

 \begin{itemize}
 \item Consecutive \verb_+_, \verb_-_, and \verb_=_ ops of
  the same type that could be combined are not allowed.
  Whether combination is possible depends on the
  attributes of the ops and whether each is multiline or
 not.  For example, two multiline deletions can never be
 consecutive, nor can any insertion come after a
 non-multiline insertion with the same attributes.
 \item ``No-op'' ops are not allowed, such as deleting 0
  characters.  However, attribute applications that don't
  have any effect are allowed.
 \item Characters at the end of the source text cannot be
  explicitly kept with no changes; if the change doesn't
  affect the last $N$ characters, those ``keep'' ops must
  be left off.
 \item In any consecutive sequence of insertions (\verb_+_)
  and deletions (\verb_-_) with no keeps (\verb_=_), the
  deletions must come before the insertions.
 \item The document text before and after will always end
  with a newline.  This policy avoids a lot of
  special-casing of the end of the document.  If a final
  newline is always added when importing text and removed
  when exporting text, then the changeset representation
  can be used to process text files that may or may not
  have a final newline.
 \end{itemize}

 \paragraph{Attribution string}

 An \emph{attribution string} is a series of inserts with
 no deletions or keeps.  For example, ``\verb_*3+8|1+5_''
 describes the attributes of a string of length 13, where
 the first 8 chars have attribute 3 and the next 5 chars
 have no attributes, with the last of these 5 chars being a
 newline.  Constraints apply similar to those affecting
 changesets, but the restriction about the final newline of
 the new document being added doesn't apply.

 Attributes in an attribution string cannot be empty, like
 ``\verb|(bold,)|'', they should instead be absent.


 \section{Further Considerations}

 \begin{itemize}
 \item composing changesets/attributions with different
  pools.
 \item generalizing ``applyToAttribution'' to make
  ``mutateAttributionLines'' and ``compose''
 \end{itemize}

 \section{Using Unicode?}

 \begin{itemize}
 \item no unicode (for efficient escaping, sightliness)
 \item efficient operations for ACE and collab (attributed text, etc.)
 \item good for time-slider
 \item good for API
 \item line-ending aware
 X more coherent (deleting or styling text merging with insertion)
 \item server-side syntax highlighting?
 \item unify author map with attribute pool
 \item unify attributed text with changeset rep
 \item not: reversible
 \item force final newline of document to be preserved
 \end{itemize}

 \paragraph{Unicode bad!}

 \begin{itemize}
 \item ugly (hard to read)
 \item more complex to parse
 \item harder to store and transmit correctly
 \item doesn't save all that much space anyway
 \item blows up in size when string-escaped
 \item embarrassing for API
 \end{itemize}


 \end{document}
--- a/doc/easysync/easysync-notes.txt
+++ b/doc/easysync/easysync-notes.txt
@ -0,0 +1,133 @@


 Copied from the old Etherpad. Found in /infrastructure/ace/

 Goals:

 - no unicode (for efficient escaping, sightliness)
 - efficient operations for ACE and collab (attributed text, etc.)
 - good for time-slider
 - good for API
 - line-ending aware
 X more coherent (deleting or styling text merging with insertion)
 - server-side syntax highlighting?
 - unify author map with attribute pool
 - unify attributed text with changeset rep
 - not: reversible
 - force final newline of document to be preserved

 - Unicode bad:
  - ugly (hard to read)
  - more complex to parse
  - harder to store and transmit correctly
  - doesn't save all that much space anyway
  - blows up in size when string-escaped
  - embarrassing for API


 # Attributes:

 An "attribute" is a (key,value) pair such as (author,abc123456) or
 (bold,true).  Sometimes an attribute is treated as an instruction to
 add that attribute, in which case an empty value means to remove it.
 So (bold,) removes the "bold" attribute.  Attributes are interned and
 given numeric IDs, so the number "6" could represent "(bold,true)",
 for example.  This mapping is stored in an attribute "pool" which may
 be shared by multiple changesets.

 Entries in the pool must be unique, so that attributes can be compared
 by their IDs.  Attribute names cannot contain commas.

 A changeset looks something like the following:

 Z:5g>1|5=2p=v*4*5+1$x

 With the corresponding pool containing these entries:

 ...
 4 -> (author,1059348573)
 5 -> (bold,true)
 ...

 This changeset, together with the pool, represents inserting
 a bold letter "x" into the middle of a line.  The string consists of:

 - a letter Z (the "magic character" and format version identifier)
 - a series of opcodes (punctuation) and numeric values in base 36 (the
  alphanumerics)
 - a dollar sign ($)
 - a string of characters used by insertion operations (the "char bank")

 If we separate out the operations and convert the numbers to base 10, we get:

 Z :196 >1 |5=97 =31 *4 *5 +1 $"x"

 Here are descriptions of the operations, where capital letters are variables:

 ":N" : Source text has length N (must be first op)
 ">N" : Final text is N (positive) characters longer than source text (must be second op)
 "<N" : Final text is N (positive) characters shorter than source text (must be second op)
 ">0" : Final text is same length as source text
 "+N" : Insert N characters from the bank, none of them newlines
 "-N" : Skip over (delete) N characters from the source text, none of them newlines
 "=N" : Keep N characters from the source text, none of them newlines
 "|L+N" : Insert N characters from the source text, containing L newlines.  The last
         character inserted MUST be a newline, but not the (new) document's final newline.  
 "|L-N" : Delete N characters from the source text, containing L newlines. The last
         character inserted MUST be a newline, but not the (old) document's final newline.
 "|L=N" : Keep N characters from the source text, containing L newlines.  The last character
         kept MUST be a newline, and the final newline of the document is allowed.
 "*I"   : Apply attribute I from the pool to the following +, =, |+, or |= command.
         In other words, any number of * ops can come before a +, =, or | but not
         between a | and the corresponding + or =.
         If +, text is inserted having this attribute.  If =, text is kept but with
         the attribute applied as an attribute addition or removal.
         Consecutive attributes must be sorted lexically by (key,value) with key
         and value taken as strings.  It's illegal to have duplicate keys
         for (key,value) pairs that apply to the same text.  It's illegal to
         have an empty value for a key in the case of an insertion (+), the
         pair should just be omitted.

 Characters from the source text that aren't accounted for are assumed to be kept
 with the same attributes.

 Additional Constraints:

 - Consecutive +, -, and = ops of the same type that could be combined are not allowed.
  Whether combination is possible depends on the attributes of the ops and whether
  each is multiline or not.  For example, two multiline deletions can never be
  consecutive, nor can any insertion come after a non-multiline insertion with the
  same attributes.
 - "No-op" ops are not allowed, such as deleting 0 characters.  However, attribute
  applications that don't have any effect are allowed.
 - Characters at the end of the source text cannot be explicitly kept with no changes;
  if the change doesn't affect the last N characters, those "keep" ops must be left off.
 - In any consecutive sequence of insertions (+) and deletions (-) with no keeps (=),
  the deletions must come before the insertions.
 - The document text before and after will always end with a newline.  This policy avoids
  a lot of special-casing of the end of the document.  If a final newline is
  always added when importing text and removed when exporting text, then the
  changeset representation can be used to process text files that may or may not
  have a final newline.

 Attribution string:

 An "attribution string" is a series of inserts with no deletions or keeps.
 For example, "*3+8|1+5" describes the attributes of a string of length 13,
 where the first 8 chars have attribute 3 and the next 5 chars have no
 attributes, with the last of these 5 chars being a newline.  Constraints
 apply similar to those affecting changesets, but the restriction about
 the final newline of the new document being added doesn't apply.

 Attributes in an attribution string cannot be empty, like "(bold,)", they should
 instead be absent.





 -------
 Considerations:

 - composing changesets/attributions with different pools
 - generalizing "applyToAttribution" to make "mutateAttributionLines" and "compose"
--- a/doc/images/etherpad_basic.png
+++ b/doc/images/etherpad_basic.png
--- a/doc/images/etherpad_demo.gif
+++ b/doc/images/etherpad_demo.gif
--- a/doc/images/etherpad_full_features.png
+++ b/doc/images/etherpad_full_features.png
--- a/doc/images/etherpad_skin_variants.gif
+++ b/doc/images/etherpad_skin_variants.gif
--- a/doc/index.md
+++ b/doc/index.md
@ -0,0 +1,9 @@
@include documentation
@include stats
@include localization
@include docker
@include skins
@include api/api
@include plugins
@include cookies
@include database
--- a/doc/localization.md
+++ b/doc/localization.md
@ -0,0 +1,114 @@
 # Localization
 Etherpad provides a multi-language user interface, that's apart from your users' content, so users from different countries can collaborate on a single document, while still having the user interface displayed in their mother tongue.


 ## Translating
 We rely on https://translatewiki.net to handle the translation process for us, so if you'd like to help...

 1. Sign up at https://translatewiki.net
 2. Visit our [TWN project page](https://translatewiki.net/wiki/Translating:Etherpad_lite)
 3. Click on `Translate Etherpad lite interface`
 4. Choose a target language, you'd like to translate our interface to, and hit `Fetch`
 5. Start translating!

 Translations will be send back to us regularly and will eventually appear in the next release.

 ## Implementation

 ### Server-side
 `/src/locales` contains files for all supported languages which contain the translated strings. Translation files are simple `*.json` files and look like this:

 ```json
 { "pad.modals.connected": "Connecté."
 , "pad.modals.uderdup": "Ouvrir dans une nouvelle fenêtre."
 , "pad.toolbar.unindent.title": "Dèsindenter"
 , "pad.toolbar.undo.title": "Annuler (Ctrl-Z)"
 , "timeslider.pageTitle": "{{appTitle}} Curseur temporel"
 , ...
 }
 ```

 Each translation consists of a key (the id of the string that is to be translated) and the translated string. Terms in curly braces must not be touched but left as they are, since they represent a dynamically changing part of the string like a variable. Imagine a message welcoming a user: `Welcome, {{userName}}!` would be translated as `Ahoy, {{userName}}!` in pirate.

 ### Client-side
 We use a `language` cookie to save your language settings if you change them. If you don't, we autodetect your locale using information from your browser. Then, the preferred language is fed into a library called [html10n.js](https://github.com/marcelklehr/html10n.js), which loads the appropriate translations and applies them to our templates. Its features include translation params, pluralization, include rules and a nice javascript API.



 ## Localizing plugins

 ### 1. Mark the strings to translate

 In the template files of your plugin, change all hardcoded messages/strings...

 from:
 ```html
 <option value="0">Heading 1</option>
 ```
 to:
 ```html
 <option data-l10n-id="ep_heading.h1" value="0"></option>
 ```

 In the javascript files of your plugin, change all hardcoded messages/strings...

 from:
 ```js
 alert ('Chat');
 ```
 to:
 ```js
 alert(window._('pad.chat'));
 ```
 ### 2. Create translate files in the locales directory of your plugin

 * The name of the file must be the language code of the language it contains translations for (see [supported lang codes](https://joker-x.github.com/languages4translatewiki/test/); e.g. en ? English, es ? Spanish...)
 * The extension of the file must be `.json`
 * The default language is English, so your plugin should always provide `en.json`
 * In order to avoid naming conflicts, your message keys should start with the name of your plugin followed by a dot (see below)

 *ep_your-plugin/locales/en.json*
 ```
 { "ep_your-plugin.h1": "Heading 1"
 }
 ```

 *ep_your-plugin/locales/es.json*
 ```
 { "ep_your-plugin.h1": "Título 1"
 }
 ```

 Every time the http server is started, it will auto-detect your messages and merge them automatically with the core messages.

 ### Overwrite core messages

 You can overwrite Etherpad's core messages in your plugin's locale files.
 For example, if you want to replace `Chat` with `Notes`, simply add...

 *ep_your-plugin/locales/en.json*
 ```
 { "ep_your-plugin.h1": "Heading 1"
 , "pad.chat": "Notes"
 }
 ```

 ## Customization for Administrators

 As an Etherpad administrator, it is possible to overwrite core mesages as well as messages in plugins. These include error messages, labels, and user instructions. Whereas the localization in the source code is in separate files separated by locale, an administrator's custom localizations are in `settings.json` under the `customLocaleStrings` key, with each locale separated by a sub-key underneath.

 For example, let's say you want to change the text on the "New Pad" button on Etherpad's home page. If you look in `locales/en.json` (or `locales/en-gb.json`) you'll see the key for this text is `"index.newPad"`. You could add the following to `settings.json`:

 ```
  "customLocaleStrings": {
    "fr": {
      "index.newPad": "Créer un document"
    },
    "en-gb": {
      "index.newPad": "Create a document"
    },
    "en": {
      "index.newPad": "Create a document"
    }
  }
 ```
--- a/doc/plugins.md
+++ b/doc/plugins.md
@ -0,0 +1,247 @@
 # Plugins

 Etherpad allows you to extend its functionality with plugins. A plugin registers
 hooks (functions) for certain events (thus certain features) in Etherpad to
 execute its own functionality based on these events.

 Publicly available plugins can be found in the npm registry (see
 <https://npmjs.org>). Etherpad's naming convention for plugins is to prefix your
 plugins with `ep_`. So, e.g. it's `ep_flubberworms`. Thus you can install
 plugins from npm, using `npm install ep_flubberworm` in Etherpad's root
 directory.

 You can also browse to `http://yourEtherpadInstan.ce/admin/plugins`, which will
 list all installed plugins and those available on npm. It even provides
 functionality to search through all available plugins.

 ## Folder structure

 Ideally a plugin has the following folder structure:

 ```
 ep_<plugin>/
 ├ .github/
 │  └ workflows/
 │     └ npmpublish.yml  ◄─ GitHub workflow to auto-publish on push
 ├ static/
 │  ├ css/               ◄─ static .css files
 │  ├ images/            ◄─ static image files
 │  ├ js/
 │  │  └ index.js        ◄─ static client-side code
 │  └ tests/
 │     ├ backend/
 │     │  └ specs/       ◄─ backend (server) tests
 │     └ frontend/
 │        └ specs/       ◄─ frontend (client) tests
 ├ templates/            ◄─ EJS templates (.html, .js, .css, etc.)
 ├ locales/
 │  ├ en.json            ◄─ English (US) strings
 │  └ qqq.json           ◄─ optional hints for translators
 ├ .travis.yml           ◄─ Travis CI config
 ├ LICENSE
 ├ README.md
 ├ ep.json               ◄─ Etherpad plugin definition
 ├ index.js              ◄─ server-side code
 ├ package.json
 └ package-lock.json
 ```

 If your plugin includes client-side hooks, put them in `static/js/`. If you're
 adding in CSS or image files, you should put those files in `static/css/ `and
 `static/image/`, respectively, and templates go into `templates/`. Translations
 go into `locales/`. Tests go in `static/tests/backend/specs/` and
 `static/tests/frontend/specs/`.

 A Standard directory structure like this makes it easier to navigate through
 your code. That said, do note, that this is not actually *required* to make your
 plugin run. If you want to make use of our i18n system, you need to put your
 translations into `locales/`, though, in order to have them integrated. (See
 "Localization" for more info on how to localize your plugin.)

 ## Plugin definition

 Your plugin definition goes into `ep.json`. In this file you register your hook
 functions, indicate the parts of your plugin and the order of execution. (A
 documentation of all available events to hook into can be found in chapter
 [hooks](#all_hooks).)

 ```json
 {
  "parts": [
    {
      "name": "nameThisPartHoweverYouWant",
      "hooks": {
        "authenticate": "ep_<plugin>/<file>:functionName1",
        "expressCreateServer": "ep_<plugin>/<file>:functionName2"
      },
      "client_hooks": {
        "acePopulateDOMLine": "ep_<plugin>/<file>:functionName3"
      }
    }
  ]
 }
 ```

 A hook function registration maps a hook name to a hook function specification.
 The hook function specification looks like `ep_example/file.js:functionName`. It
 consists of two parts separated by a colon: a module name to `require()` and the
 name of a function exported by the named module. See
 [`module.exports`](https://nodejs.org/docs/latest/api/modules.html#modules_module_exports)
 for how to export a function.

 For the module name you can omit the `.js` suffix, and if the file is `index.js`
 you can use just the directory name. You can also omit the module name entirely,
 in which case it defaults to the plugin name (e.g., `ep_example`).

 You can also omit the function name. If you do, Etherpad will look for an
 exported function whose name matches the name of the hook (e.g.,
 `authenticate`).

 If either the module name or the function name is omitted (or both), the colon
 may also be omitted unless the provided module name contains a colon. (So if the
 module name is `C:\foo.js` then the hook function specification with the
 function name omitted would be `"C:\foo.js:"`.)

 Examples: Suppose the plugin name is `ep_example`. All of the following are
 equivalent, and will cause the `authorize` hook to call the `exports.authorize`
 function in `index.js` from the `ep_example` plugin:

 * `"authorize": "ep_example/index.js:authorize"`
 * `"authorize": "ep_example/index.js:"`
 * `"authorize": "ep_example/index.js"`
 * `"authorize": "ep_example/index:authorize"`
 * `"authorize": "ep_example/index:"`
 * `"authorize": "ep_example/index"`
 * `"authorize": "ep_example:authorize"`
 * `"authorize": "ep_example:"`
 * `"authorize": "ep_example"`
 * `"authorize": ":authorize"`
 * `"authorize": ":"`
 * `"authorize": ""`

 ### Client hooks and server hooks

 There are server hooks, which will be executed on the server (e.g.
 `expressCreateServer`), and there are client hooks, which are executed on the
 client (e.g. `acePopulateDomLine`). Be sure to not make assumptions about the
 environment your code is running in, e.g. don't try to access `process`, if you
 know your code will be run on the client, and likewise, don't try to access
 `window` on the server...

 ### Styling

 When you install a client-side plugin (e.g. one that implements at least one
 client-side hook), the plugin name is added to the `class` attribute of the div
 `#editorcontainerbox` in the main window. This gives you the opportunity of
 tuning the appearance of the main UI in your plugin.

 For example, this is the markup with no plugins installed:

 ```html
 <div id="editorcontainerbox" class="">
 ```

 and this is the contents after installing `someplugin`:

 ```html
 <div id="editorcontainerbox" class="ep_someplugin">
 ```

 This feature was introduced in Etherpad **1.8**.

 ### Parts

 As your plugins become more and more complex, you will find yourself in the need
 to manage dependencies between plugins. E.g. you want the hooks of a certain
 plugin to be executed before (or after) yours. You can also manage these
 dependencies in your plugin definition file `ep.json`:

 ```json
 {
  "parts": [
    {
      "name": "onepart",
      "pre": [],
      "post": ["ep_onemoreplugin/partone"]
      "hooks": {
        "storeBar": "ep_monospace/plugin:storeBar",
        "getFoo": "ep_monospace/plugin:getFoo",
      }
    },
    {
      "name": "otherpart",
      "pre": ["ep_my_example/somepart", "ep_otherplugin/main"],
      "post": [],
      "hooks": {
        "someEvent": "ep_my_example/otherpart:someEvent",
        "another": "ep_my_example/otherpart:another"
      }
    }
  ]
 }
 ```

 Usually a plugin will add only one functionality at a time, so it will probably
 only use one `part` definition to register its hooks. However, sometimes you
 have to put different (unrelated) functionalities into one plugin. For this you
 will want use parts, so other plugins can depend on them.

 #### pre/post

 The `"pre"` and `"post"` definitions, affect the order in which parts of a
 plugin are executed. This ensures that plugins and their hooks are executed in
 the correct order.

 `"pre"` lists parts that must be executed *before* the defining part. `"post"`
 lists parts that must be executed *after* the defining part.

 You can, on a basic level, think of this as double-ended dependency listing. If
 you have a dependency on another plugin, you can make sure it loads before yours
 by putting it in `"pre"`. If you are setting up things that might need to be
 used by a plugin later, you can ensure proper order by putting it in `"post"`.

 Note that it would be far more sane to use `"pre"` in almost any case, but if
 you want to change config variables for another plugin, or maybe modify its
 environment, `"post"` could definitely be useful.

 Also, note that dependencies should *also* be listed in your package.json, so
 they can be `npm install`'d automagically when your plugin gets installed.

 ## Package definition

 Your plugin must also contain a [package definition
 file](https://docs.npmjs.com/files/package.json), called package.json, in the
 project root - this file contains various metadata relevant to your plugin, such
 as the name and version number, author, project hompage, contributors, a short
 description, etc. If you publish your plugin on npm, these metadata are used for
 package search etc., but it's necessary for Etherpad plugins, even if you don't
 publish your plugin.

 ```json
 {
  "name": "ep_PLUGINNAME",
  "version": "0.0.1",
  "description": "DESCRIPTION",
  "author": "USERNAME (REAL NAME) <MAIL@EXAMPLE.COM>",
  "contributors": [],
  "dependencies": {"MODULE": "0.3.20"},
  "engines": { "node": ">= 10.13.0"}
 }
 ```

 ## Templates

 If your plugin adds or modifies the front end HTML (e.g. adding buttons or
 changing their functions), you should put the necessary HTML code for such
 operations in `templates/`, in files of type ".ejs", since Etherpad uses EJS for
 HTML templating. See the following link for more information about EJS:
 <https://github.com/visionmedia/ejs>.

 ## Writing and running front-end tests for your plugin

 Etherpad allows you to easily create front-end tests for plugins.

 1. Create a new folder: `%your_plugin%/static/tests/frontend/specs`
 2. Put your spec file in there. (Example spec files are visible in
   `%etherpad_root_folder%/frontend/tests/specs`.)
 3. Visit http://yourserver.com/frontend/tests and your front-end tests will run.
--- a/doc/skins.md
+++ b/doc/skins.md
@ -0,0 +1,19 @@
 # Skins
 You can customize Etherpad appearance using skins.
 A skin is a directory located under `static/skins/<skin_name>`, with the following contents:

 * `index.js`: javascript that will be run in `/`
 * `index.css`: stylesheet affecting `/`
 * `pad.js`: javascript that will be run in `/p/:padid`
 * `pad.css`: stylesheet affecting `/p/:padid`
 * `timeslider.js`: javascript that will be run in `/p/:padid/timeslider`
 * `timeslider.css`: stylesheet affecting `/p/:padid/timeslider`
 * `favicon.ico`: overrides the default favicon
 * `robots.txt`: overrides the default `robots.txt`

 You can choose a skin changing the parameter `skinName` in `settings.json`.

 Since Etherpad **1.7.5**, two skins are included:

 * `no-skin`: an empty skin, leaving the default Etherpad appearance unchanged, that you can use as a guidance to develop your own.
 * `colibris`: a new, experimental skin, that will become the default in Etherpad 2.0.
--- a/doc/stats.md
+++ b/doc/stats.md
@ -0,0 +1,18 @@
 # Statistics
 Etherpad keeps track of the goings-on inside the edit machinery. If you'd like to have a look at this, just point your browser to `/stats`.

 We currently measure:

 - totalUsers (counter)
 - connects (meter)
 - disconnects (meter)
 - pendingEdits (counter)
 - edits (timer)
 - failedChangesets (meter)
 - httpRequests (timer)
 - http500 (meter)
 - memoryUsage (gauge)

 Under the hood, we are happy to rely on [measured](https://github.com/felixge/node-measured) for all our metrics needs.

 To modify or simply access our stats in your plugin, simply `require('ep_etherpad-lite/stats')` which is a [`measured.Collection`](https://yaorg.github.io/node-measured/packages/measured-core/Collection.html).
--- a/doc/template.html
+++ b/doc/template.html
@ -0,0 +1,23 @@
 <!doctype html>
 <html lang="en">
 <head>
  <meta charset="utf-8">
  <title>__SECTION__ - Etherpad v__VERSION__ Manual &amp; Documentation</title>
  <link rel="stylesheet" href="assets/style.css">
 </head>
 <body class="apidoc" id="api-section-__FILENAME__">
  <header id="header">
    <h1>Etherpad v__VERSION__ Manual &amp; Documentation</h1>
  </header>

  <div id="toc">
    <h2>Table of Contents</h2>
    __TOC__
  </div>

  <div id="apicontent">
    __CONTENT__
  </div>

 </body>
 </html>
--- a/node_modules/.bin/acorn
+++ b/node_modules/.bin/acorn
@ -0,0 +1,4 @@
 #!/usr/bin/env node
 'use strict';

 require('../dist/bin.js');
--- a/node_modules/.bin/eslint
+++ b/node_modules/.bin/eslint
@ -0,0 +1,146 @@
 #!/usr/bin/env node

 /**
 * @fileoverview Main CLI that is run via the eslint command.
 * @author Nicholas C. Zakas
 */

 /* eslint no-console:off */

 "use strict";

 // to use V8's code cache to speed up instantiation time
 require("v8-compile-cache");

 // must do this initialization *before* other requires in order to work
 if (process.argv.includes("--debug")) {
    require("debug").enable("eslint:*,-eslint:code-path,eslintrc:*");
 }

 //------------------------------------------------------------------------------
 // Helpers
 //------------------------------------------------------------------------------

 /**
 * Read data from stdin til the end.
 *
 * Note: See
 * - https://github.com/nodejs/node/blob/master/doc/api/process.md#processstdin
 * - https://github.com/nodejs/node/blob/master/doc/api/process.md#a-note-on-process-io
 * - https://lists.gnu.org/archive/html/bug-gnu-emacs/2016-01/msg00419.html
 * - https://github.com/nodejs/node/issues/7439 (historical)
 *
 * On Windows using `fs.readFileSync(STDIN_FILE_DESCRIPTOR, "utf8")` seems
 * to read 4096 bytes before blocking and never drains to read further data.
 *
 * The investigation on the Emacs thread indicates:
 *
 * > Emacs on MS-Windows uses pipes to communicate with subprocesses; a
 * > pipe on Windows has a 4K buffer. So as soon as Emacs writes more than
 * > 4096 bytes to the pipe, the pipe becomes full, and Emacs then waits for
 * > the subprocess to read its end of the pipe, at which time Emacs will
 * > write the rest of the stuff.
 * @returns {Promise<string>} The read text.
 */
 function readStdin() {
    return new Promise((resolve, reject) => {
        let content = "";
        let chunk = "";

        process.stdin
            .setEncoding("utf8")
            .on("readable", () => {
                while ((chunk = process.stdin.read()) !== null) {
                    content += chunk;
                }
            })
            .on("end", () => resolve(content))
            .on("error", reject);
    });
 }

 /**
 * Get the error message of a given value.
 * @param {any} error The value to get.
 * @returns {string} The error message.
 */
 function getErrorMessage(error) {

    // Lazy loading because those are used only if error happened.
    const fs = require("fs");
    const path = require("path");
    const util = require("util");
    const lodash = require("lodash");

    // Foolproof -- thirdparty module might throw non-object.
    if (typeof error !== "object" || error === null) {
        return String(error);
    }

    // Use templates if `error.messageTemplate` is present.
    if (typeof error.messageTemplate === "string") {
        try {
            const templateFilePath = path.resolve(
                __dirname,
                `../messages/${error.messageTemplate}.txt`
            );

            // Use sync API because Node.js should exit at this tick.
            const templateText = fs.readFileSync(templateFilePath, "utf-8");
            const template = lodash.template(templateText);

            return template(error.messageData || {});
        } catch {

            // Ignore template error then fallback to use `error.stack`.
        }
    }

    // Use the stacktrace if it's an error object.
    if (typeof error.stack === "string") {
        return error.stack;
    }

    // Otherwise, dump the object.
    return util.format("%o", error);
 }

 /**
 * Catch and report unexpected error.
 * @param {any} error The thrown error object.
 * @returns {void}
 */
 function onFatalError(error) {
    process.exitCode = 2;

    const { version } = require("../package.json");
    const message = getErrorMessage(error);

    console.error(`
 Oops! Something went wrong! :(

 ESLint: ${version}

 ${message}`);
 }

 //------------------------------------------------------------------------------
 // Execution
 //------------------------------------------------------------------------------

 (async function main() {
    process.on("uncaughtException", onFatalError);
    process.on("unhandledRejection", onFatalError);

    // Call the config initializer if `--init` is present.
    if (process.argv.includes("--init")) {
        await require("../lib/init/config-initializer").initializeConfig();
        return;
    }

    // Otherwise, call the CLI.
    process.exitCode = await require("../lib/cli").execute(
        process.argv,
        process.argv.includes("--stdin") ? await readStdin() : null
    );
 }()).catch(onFatalError);
--- a/node_modules/.bin/esparse
+++ b/node_modules/.bin/esparse
@ -0,0 +1,139 @@
 #!/usr/bin/env node
 /*
  Copyright JS Foundation and other contributors, https://js.foundation/

  Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.

  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
  AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  ARE DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
  ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
  THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

 /*jslint sloppy:true node:true rhino:true */

 var fs, esprima, fname, forceFile, content, options, syntax;

 if (typeof require === 'function') {
    fs = require('fs');
    try {
        esprima = require('esprima');
    } catch (e) {
        esprima = require('../');
    }
 } else if (typeof load === 'function') {
    try {
        load('esprima.js');
    } catch (e) {
        load('../esprima.js');
    }
 }

 // Shims to Node.js objects when running under Rhino.
 if (typeof console === 'undefined' && typeof process === 'undefined') {
    console = { log: print };
    fs = { readFileSync: readFile };
    process = { argv: arguments, exit: quit };
    process.argv.unshift('esparse.js');
    process.argv.unshift('rhino');
 }

 function showUsage() {
    console.log('Usage:');
    console.log('   esparse [options] [file.js]');
    console.log();
    console.log('Available options:');
    console.log();
    console.log('  --comment      Gather all line and block comments in an array');
    console.log('  --loc          Include line-column location info for each syntax node');
    console.log('  --range        Include index-based range for each syntax node');
    console.log('  --raw          Display the raw value of literals');
    console.log('  --tokens       List all tokens in an array');
    console.log('  --tolerant     Tolerate errors on a best-effort basis (experimental)');
    console.log('  -v, --version  Shows program version');
    console.log();
    process.exit(1);
 }

 options = {};

 process.argv.splice(2).forEach(function (entry) {

    if (forceFile || entry === '-' || entry.slice(0, 1) !== '-') {
        if (typeof fname === 'string') {
            console.log('Error: more than one input file.');
            process.exit(1);
        } else {
            fname = entry;
        }
    } else if (entry === '-h' || entry === '--help') {
        showUsage();
    } else if (entry === '-v' || entry === '--version') {
        console.log('ECMAScript Parser (using Esprima version', esprima.version, ')');
        console.log();
        process.exit(0);
    } else if (entry === '--comment') {
        options.comment = true;
    } else if (entry === '--loc') {
        options.loc = true;
    } else if (entry === '--range') {
        options.range = true;
    } else if (entry === '--raw') {
        options.raw = true;
    } else if (entry === '--tokens') {
        options.tokens = true;
    } else if (entry === '--tolerant') {
        options.tolerant = true;
    } else if (entry === '--') {
        forceFile = true;
    } else {
        console.log('Error: unknown option ' + entry + '.');
        process.exit(1);
    }
 });

 // Special handling for regular expression literal since we need to
 // convert it to a string literal, otherwise it will be decoded
 // as object "{}" and the regular expression would be lost.
 function adjustRegexLiteral(key, value) {
    if (key === 'value' && value instanceof RegExp) {
        value = value.toString();
    }
    return value;
 }

 function run(content) {
    syntax = esprima.parse(content, options);
    console.log(JSON.stringify(syntax, adjustRegexLiteral, 4));
 }

 try {
    if (fname && (fname !== '-' || forceFile)) {
        run(fs.readFileSync(fname, 'utf-8'));
    } else {
        var content = '';
        process.stdin.resume();
        process.stdin.on('data', function(chunk) {
            content += chunk;
        });
        process.stdin.on('end', function() {
            run(content);
        });
    }
 } catch (e) {
    console.log('Error: ' + e.message);
    process.exit(1);
 }
--- a/node_modules/.bin/esvalidate
+++ b/node_modules/.bin/esvalidate
@ -0,0 +1,236 @@
 #!/usr/bin/env node
 /*
  Copyright JS Foundation and other contributors, https://js.foundation/

  Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.

  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
  AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  ARE DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
  ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
  THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

 /*jslint sloppy:true plusplus:true node:true rhino:true */
 /*global phantom:true */

 var fs, system, esprima, options, fnames, forceFile, count;

 if (typeof esprima === 'undefined') {
    // PhantomJS can only require() relative files
    if (typeof phantom === 'object') {
        fs = require('fs');
        system = require('system');
        esprima = require('./esprima');
    } else if (typeof require === 'function') {
        fs = require('fs');
        try {
            esprima = require('esprima');
        } catch (e) {
            esprima = require('../');
        }
    } else if (typeof load === 'function') {
        try {
            load('esprima.js');
        } catch (e) {
            load('../esprima.js');
        }
    }
 }

 // Shims to Node.js objects when running under PhantomJS 1.7+.
 if (typeof phantom === 'object') {
    fs.readFileSync = fs.read;
    process = {
        argv: [].slice.call(system.args),
        exit: phantom.exit,
        on: function (evt, callback) {
            callback();
        }
    };
    process.argv.unshift('phantomjs');
 }

 // Shims to Node.js objects when running under Rhino.
 if (typeof console === 'undefined' && typeof process === 'undefined') {
    console = { log: print };
    fs = { readFileSync: readFile };
    process = {
        argv: arguments,
        exit: quit,
        on: function (evt, callback) {
            callback();
        }
    };
    process.argv.unshift('esvalidate.js');
    process.argv.unshift('rhino');
 }

 function showUsage() {
    console.log('Usage:');
    console.log('   esvalidate [options] [file.js...]');
    console.log();
    console.log('Available options:');
    console.log();
    console.log('  --format=type  Set the report format, plain (default) or junit');
    console.log('  -v, --version  Print program version');
    console.log();
    process.exit(1);
 }

 options = {
    format: 'plain'
 };

 fnames = [];

 process.argv.splice(2).forEach(function (entry) {

    if (forceFile || entry === '-' || entry.slice(0, 1) !== '-') {
        fnames.push(entry);
    } else if (entry === '-h' || entry === '--help') {
        showUsage();
    } else if (entry === '-v' || entry === '--version') {
        console.log('ECMAScript Validator (using Esprima version', esprima.version, ')');
        console.log();
        process.exit(0);
    } else if (entry.slice(0, 9) === '--format=') {
        options.format = entry.slice(9);
        if (options.format !== 'plain' && options.format !== 'junit') {
            console.log('Error: unknown report format ' + options.format + '.');
            process.exit(1);
        }
    } else if (entry === '--') {
        forceFile = true;
    } else {
        console.log('Error: unknown option ' + entry + '.');
        process.exit(1);
    }
 });

 if (fnames.length === 0) {
    fnames.push('');
 }

 if (options.format === 'junit') {
    console.log('<?xml version="1.0" encoding="UTF-8"?>');
    console.log('<testsuites>');
 }

 count = 0;

 function run(fname, content) {
    var timestamp, syntax, name;
    try {
        if (typeof content !== 'string') {
            throw content;
        }

        if (content[0] === '#' && content[1] === '!') {
            content = '//' + content.substr(2, content.length);
        }

        timestamp = Date.now();
        syntax = esprima.parse(content, { tolerant: true });

        if (options.format === 'junit') {

            name = fname;
            if (name.lastIndexOf('/') >= 0) {
                name = name.slice(name.lastIndexOf('/') + 1);
            }

            console.log('<testsuite name="' + fname + '" errors="0" ' +
                ' failures="' + syntax.errors.length + '" ' +
                ' tests="' + syntax.errors.length + '" ' +
                ' time="' + Math.round((Date.now() - timestamp) / 1000) +
                '">');

            syntax.errors.forEach(function (error) {
                var msg = error.message;
                msg = msg.replace(/^Line\ [0-9]*\:\ /, '');
                console.log('  <testcase name="Line ' + error.lineNumber + ': ' + msg + '" ' +
                    ' time="0">');
                console.log('    <error type="SyntaxError" message="' + error.message + '">' +
                    error.message + '(' + name + ':' + error.lineNumber + ')' +
                    '</error>');
                console.log('  </testcase>');
            });

            console.log('</testsuite>');

        } else if (options.format === 'plain') {

            syntax.errors.forEach(function (error) {
                var msg = error.message;
                msg = msg.replace(/^Line\ [0-9]*\:\ /, '');
                msg = fname + ':' + error.lineNumber + ': ' + msg;
                console.log(msg);
                ++count;
            });

        }
    } catch (e) {
        ++count;
        if (options.format === 'junit') {
            console.log('<testsuite name="' + fname + '" errors="1" failures="0" tests="1" ' +
                ' time="' + Math.round((Date.now() - timestamp) / 1000) + '">');
            console.log(' <testcase name="' + e.message + '" ' + ' time="0">');
            console.log(' <error type="ParseError" message="' + e.message + '">' +
                e.message + '(' + fname + ((e.lineNumber) ? ':' + e.lineNumber : '') +
                ')</error>');
            console.log(' </testcase>');
            console.log('</testsuite>');
        } else {
            console.log(fname + ':' + e.lineNumber + ': ' + e.message.replace(/^Line\ [0-9]*\:\ /, ''));
        }
    }
 }

 fnames.forEach(function (fname) {
    var content = '';
    try {
        if (fname && (fname !== '-' || forceFile)) {
            content = fs.readFileSync(fname, 'utf-8');
        } else {
            fname = '';
            process.stdin.resume();
            process.stdin.on('data', function(chunk) {
                content += chunk;
            });
            process.stdin.on('end', function() {
                run(fname, content);
            });
            return;
        }
    } catch (e) {
        content = e;
    }
    run(fname, content);
 });

 process.on('exit', function () {
    if (options.format === 'junit') {
        console.log('</testsuites>');
    }

    if (count > 0) {
        process.exit(1);
    }

    if (count === 0 && typeof phantom === 'object') {
        process.exit(0);
    }
 });
--- a/node_modules/.bin/js-yaml
+++ b/node_modules/.bin/js-yaml
@ -0,0 +1,132 @@
 #!/usr/bin/env node


 'use strict';

 /*eslint-disable no-console*/


 // stdlib
 var fs    = require('fs');


 // 3rd-party
 var argparse = require('argparse');


 // internal
 var yaml = require('..');


 ////////////////////////////////////////////////////////////////////////////////


 var cli = new argparse.ArgumentParser({
  prog:     'js-yaml',
  version:  require('../package.json').version,
  addHelp:  true
 });


 cli.addArgument([ '-c', '--compact' ], {
  help:   'Display errors in compact mode',
  action: 'storeTrue'
 });


 // deprecated (not needed after we removed output colors)
 // option suppressed, but not completely removed for compatibility
 cli.addArgument([ '-j', '--to-json' ], {
  help:   argparse.Const.SUPPRESS,
  dest:   'json',
  action: 'storeTrue'
 });


 cli.addArgument([ '-t', '--trace' ], {
  help:   'Show stack trace on error',
  action: 'storeTrue'
 });

 cli.addArgument([ 'file' ], {
  help:   'File to read, utf-8 encoded without BOM',
  nargs:  '?',
  defaultValue: '-'
 });


 ////////////////////////////////////////////////////////////////////////////////


 var options = cli.parseArgs();


 ////////////////////////////////////////////////////////////////////////////////

 function readFile(filename, encoding, callback) {
  if (options.file === '-') {
    // read from stdin

    var chunks = [];

    process.stdin.on('data', function (chunk) {
      chunks.push(chunk);
    });

    process.stdin.on('end', function () {
      return callback(null, Buffer.concat(chunks).toString(encoding));
    });
  } else {
    fs.readFile(filename, encoding, callback);
  }
 }

 readFile(options.file, 'utf8', function (error, input) {
  var output, isYaml;

  if (error) {
    if (error.code === 'ENOENT') {
      console.error('File not found: ' + options.file);
      process.exit(2);
    }

    console.error(
      options.trace && error.stack ||
      error.message ||
      String(error));

    process.exit(1);
  }

  try {
    output = JSON.parse(input);
    isYaml = false;
  } catch (err) {
    if (err instanceof SyntaxError) {
      try {
        output = [];
        yaml.loadAll(input, function (doc) { output.push(doc); }, {});
        isYaml = true;

        if (output.length === 0) output = null;
        else if (output.length === 1) output = output[0];

      } catch (e) {
        if (options.trace && err.stack) console.error(e.stack);
        else console.error(e.toString(options.compact));

        process.exit(1);
      }
    } else {
      console.error(
        options.trace && err.stack ||
        err.message ||
        String(err));

      process.exit(1);
    }
  }

  if (isYaml) console.log(JSON.stringify(output, null, '  '));
  else console.log(yaml.dump(output));
 });
--- a/node_modules/.bin/node-which
+++ b/node_modules/.bin/node-which
@ -0,0 +1,52 @@
 #!/usr/bin/env node
 var which = require("../")
 if (process.argv.length < 3)
  usage()

 function usage () {
  console.error('usage: which [-as] program ...')
  process.exit(1)
 }

 var all = false
 var silent = false
 var dashdash = false
 var args = process.argv.slice(2).filter(function (arg) {
  if (dashdash || !/^-/.test(arg))
    return true

  if (arg === '--') {
    dashdash = true
    return false
  }

  var flags = arg.substr(1).split('')
  for (var f = 0; f < flags.length; f++) {
    var flag = flags[f]
    switch (flag) {
      case 's':
        silent = true
        break
      case 'a':
        all = true
        break
      default:
        console.error('which: illegal option -- ' + flag)
        usage()
    }
  }
  return false
 })

 process.exit(args.reduce(function (pv, current) {
  try {
    var f = which.sync(current, { all: all })
    if (all)
      f = f.join('\n')
    if (!silent)
      console.log(f)
    return pv;
  } catch (e) {
    return 1;
  }
 }, 0))
--- a/node_modules/.bin/rimraf
+++ b/node_modules/.bin/rimraf
@ -0,0 +1,68 @@
 #!/usr/bin/env node

 const rimraf = require('./')

 const path = require('path')

 const isRoot = arg => /^(\/|[a-zA-Z]:\)$/.test(path.resolve(arg))
 const filterOutRoot = arg => {
  const ok = preserveRoot === false || !isRoot(arg)
  if (!ok) {
    console.error(`refusing to remove ${arg}`)
    console.error('Set --no-preserve-root to allow this')
  }
  return ok
 }

 let help = false
 let dashdash = false
 let noglob = false
 let preserveRoot = true
 const args = process.argv.slice(2).filter(arg => {
  if (dashdash)
    return !!arg
  else if (arg === '--')
    dashdash = true
  else if (arg === '--no-glob' || arg === '-G')
    noglob = true
  else if (arg === '--glob' || arg === '-g')
    noglob = false
  else if (arg.match(/^(-+|\/)(h(elp)?|\?)$/))
    help = true
  else if (arg === '--preserve-root')
    preserveRoot = true
  else if (arg === '--no-preserve-root')
    preserveRoot = false
  else
    return !!arg
 }).filter(arg => !preserveRoot || filterOutRoot(arg))

 const go = n => {
  if (n >= args.length)
    return
  const options = noglob ? { glob: false } : {}
  rimraf(args[n], options, er => {
    if (er)
      throw er
    go(n+1)
  })
 }

 if (help || args.length === 0) {
  // If they didn't ask for help, then this is not a "success"
  const log = help ? console.log : console.error
  log('Usage: rimraf <path> [<path> ...]')
  log('')
  log('  Deletes all files and folders at "path" recursively.')
  log('')
  log('Options:')
  log('')
  log('  -h, --help          Display this usage info')
  log('  -G, --no-glob       Do not expand glob patterns in arguments')
  log('  -g, --glob          Expand glob patterns in arguments (default)')
  log('  --preserve-root     Do not remove \'/\' (default)')
  log('  --no-preserve-root  Do not treat \'/\' specially')
  log('  --                  Stop parsing flags')
  process.exit(help ? 0 : 1)
 } else
  go(0)
--- a/node_modules/.bin/semver
+++ b/node_modules/.bin/semver
@ -0,0 +1,173 @@
 #!/usr/bin/env node
 // Standalone semver comparison program.
 // Exits successfully and prints matching version(s) if
 // any supplied version is valid and passes all tests.

 const argv = process.argv.slice(2)

 let versions = []

 const range = []

 let inc = null

 const version = require('../package.json').version

 let loose = false

 let includePrerelease = false

 let coerce = false

 let rtl = false

 let identifier

 const semver = require('../')

 let reverse = false

 const options = {}

 const main = () => {
  if (!argv.length) return help()
  while (argv.length) {
    let a = argv.shift()
    const indexOfEqualSign = a.indexOf('=')
    if (indexOfEqualSign !== -1) {
      a = a.slice(0, indexOfEqualSign)
      argv.unshift(a.slice(indexOfEqualSign + 1))
    }
    switch (a) {
      case '-rv': case '-rev': case '--rev': case '--reverse':
        reverse = true
        break
      case '-l': case '--loose':
        loose = true
        break
      case '-p': case '--include-prerelease':
        includePrerelease = true
        break
      case '-v': case '--version':
        versions.push(argv.shift())
        break
      case '-i': case '--inc': case '--increment':
        switch (argv[0]) {
          case 'major': case 'minor': case 'patch': case 'prerelease':
          case 'premajor': case 'preminor': case 'prepatch':
            inc = argv.shift()
            break
          default:
            inc = 'patch'
            break
        }
        break
      case '--preid':
        identifier = argv.shift()
        break
      case '-r': case '--range':
        range.push(argv.shift())
        break
      case '-c': case '--coerce':
        coerce = true
        break
      case '--rtl':
        rtl = true
        break
      case '--ltr':
        rtl = false
        break
      case '-h': case '--help': case '-?':
        return help()
      default:
        versions.push(a)
        break
    }
  }

  const options = { loose: loose, includePrerelease: includePrerelease, rtl: rtl }

  versions = versions.map((v) => {
    return coerce ? (semver.coerce(v, options) || { version: v }).version : v
  }).filter((v) => {
    return semver.valid(v)
  })
  if (!versions.length) return fail()
  if (inc && (versions.length !== 1 || range.length)) { return failInc() }

  for (let i = 0, l = range.length; i < l; i++) {
    versions = versions.filter((v) => {
      return semver.satisfies(v, range[i], options)
    })
    if (!versions.length) return fail()
  }
  return success(versions)
 }


 const failInc = () => {
  console.error('--inc can only be used on a single version with no range')
  fail()
 }

 const fail = () => process.exit(1)

 const success = () => {
  const compare = reverse ? 'rcompare' : 'compare'
  versions.sort((a, b) => {
    return semver[compare](a, b, options)
  }).map((v) => {
    return semver.clean(v, options)
  }).map((v) => {
    return inc ? semver.inc(v, inc, options, identifier) : v
  }).forEach((v, i, _) => { console.log(v) })
 }

 const help = () => console.log(
 `SemVer ${version}

 A JavaScript implementation of the https://semver.org/ specification
 Copyright Isaac Z. Schlueter

 Usage: semver [options] <version> [<version> [...]]
 Prints valid versions sorted by SemVer precedence

 Options:
 -r --range <range>
        Print versions that match the specified range.

 -i --increment [<level>]
        Increment a version by the specified level.  Level can
        be one of: major, minor, patch, premajor, preminor,
        prepatch, or prerelease.  Default level is 'patch'.
        Only one version may be specified.

 --preid <identifier>
        Identifier to be used to prefix premajor, preminor,
        prepatch or prerelease version increments.

 -l --loose
        Interpret versions and ranges loosely

 -p --include-prerelease
        Always include prerelease versions in range matching

 -c --coerce
        Coerce a string into SemVer if possible
        (does not imply --loose)

 --rtl
        Coerce version strings right to left

 --ltr
        Coerce version strings left to right (default)

 Program exits successfully if any valid version satisfies
 all supplied ranges, and prints all satisfying versions.

 If no satisfying versions are found, then exits failure.

 Versions are printed in ascending order, so supplying
 multiple versions to the utility will just sort them.`)

 main()