Skip to content

Groupware#1994

Open
pbleser-oc wants to merge 278 commits into
mainfrom
groupware
Open

Groupware#1994
pbleser-oc wants to merge 278 commits into
mainfrom
groupware

Conversation

@pbleser-oc

@pbleser-oc pbleser-oc commented Dec 9, 2025

Copy link
Copy Markdown
Member

Description

Ongoing implementation of the Groupware backend service, which initially happened on its own branch but should now occur on the main branch.

Changes are almost exclusively on packages that are distinct to the Groupware backend, namely

  • pkg/jmap, pkg/jscalendar and pkg/jscontact: contain the implementation of the JMAP protocol and data models for Core, Mail, Contacts, Calendars, Blobs, ...
  • services/groupware: contains the Groupware backend service which is currently configured to not be started by default, and must be explicitly included in the START_ADDITIONAL_SERVICES property

Changes to common areas include:

  • devtools/deployments/opencloud_full: addition of a container of the Stalwart mail server which is used for Groupware functionality
  • pkg/structs: add more helper functions that are then used in pkg/jmap and services/groupware

Note that it is not fully functional yet and is going to be under continued and ongoing development along with its UI counterparts.

Specifically, the following aspects are lacking and only implemented in a skeletal way as a proof of concept:

  • API documentation: originally developed using go-swagger and redocly-cli, but which is having severe limitations and might be superseded by an ongoing implementation of a custom source code parser which renders OpenAPI
  • unit tests: very little coverage if at all, which is mostly due to most of the implementation of the Groupware backend doing relatively little logic and is mostly implementing the JMAP model
  • integration tests: some coverage exists that performs tests against a Stalwart container using testcontainers, but will be made more exhaustive
  • k6 black box tests: none implemented yet
  • system documentation: none implemented yet, too early to do so

The Groupware backend is not meant to be used productively yet.

@pbleser-oc pbleser-oc self-assigned this Dec 9, 2025
@pbleser-oc pbleser-oc requested a review from butonic December 10, 2025 08:32
@pbleser-oc pbleser-oc marked this pull request as ready for review December 10, 2025 08:32
Comment thread services/auth-api/pkg/service/http/v0/service.go Outdated
Comment thread services/groupware/DEVELOPER.md Outdated
Comment thread services/groupware/DEVELOPER.md Outdated
curl -ks -D- -X POST \
"https://keycloak.opencloud.test/realms/openCloud/protocol/openid-connect/token" \
-d username=alan -d password=demo -d grant_type=password \
-d client_id=groupware -d scope=openid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when working from top to bottom this fails. using a client_id=web works ... so I need to add a client id for the goupware somewhere ...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did add a JSON file devtools/deployments/opencloud_full/config/keycloak/clients/groupware.json, expecting it to be automatically picked up, but that might require re-creating the Keycloak container... ?
In any case, that was a beginner's mistake, there is really no reason to have a distinct client ID for the groupware, as the Web UI is going to use web instead.

Removed that config file and updated the documentation accordingly to use web instead, in cfbbe02

to which one should receive the following response:

```java
A OK [CAPABILITY IMAP4rev2 ...] Authentication successful

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, I need to dig into:

stalwart-1            | 2025-12-10T16:07:05Z TRACE Raw IMAP input received (imap.raw-input) listenerId = "imaptls", localPort = 993, remoteIp = 172.39.0.1, remotePort = 45034, size = 19, contents = "A LOGIN alan demo\r\n"
stalwart-1            | 2025-12-10T16:07:05Z ERROR LDAP error (store.ldap-error) listenerId = "imaptls", localPort = 993, remoteIp = 172.39.0.1, remotePort = 45034, reason = "I/O error: Connection refused (os error 111)", causedBy = "crates/directory/src/core/dispatch.rs:25", id = "A"

Comment thread devtools/deployments/opencloud_full/.env
Comment thread services/groupware/DEVELOPER.md
Comment thread services/groupware/pkg/groupware/api.go
Comment thread pkg/jmap/api.go
@@ -0,0 +1,49 @@
package jmap

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should drop the jmap_ prefix from all files in this package

AUTH_BASIC_LDAP_BIND_PASSWORD: "admin"
USERS_LDAP_BIND_PASSWORD: "admin"
GROUPS_LDAP_BIND_PASSWORD: "admin"
IDM_LDAPS_ADDR: 0.0.0.0:9235

@butonic butonic Dec 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I configured everything on .compose.test to prevent collisions with my .opencloud.test deployment. The stalwart URL then needs to be set for the groupware:

Suggested change
IDM_LDAPS_ADDR: 0.0.0.0:9235
IDM_LDAPS_ADDR: 0.0.0.0:9235
GROUPWARE_JMAP_BASE_URL: https://${STALWART_DOMAIN:-stalwart.opencloud.test}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh and we need to set FRONTEND_GROUPWARE_ENABLED: "true" and enable the mail app in the web config when stalwart is enabled ... but that connot be configured with a simple env var ... 😞 we need to replace the config file then ...

@butonic butonic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I still don't have a menu entry for mail in the web UI even though the config.json contains the mail app. I assume that is a problem with my setup/config. in any case, I can see the mail UI when navigation to {$OC_URL}/mail manually. \o/

I did run into some minor issues when following the docs. If you could address them (and enlighten me on how to get menu to show the groupware apps) I'm happy to merge this. I mostly want others to be able follow the DEVELOPMENT.md and have a working setup. Kudos for that, btw.

The groupware service itself follows our ... boilerplate ... service code ... and implements the JMAP handling. I assume that will have to evolve, but we can merge it and iterate.

Tip

the web repo has a compose file with all the apps enabled. That gave me the final hint to get the menu entries

pbleser-oc added 29 commits July 3, 2026 15:40
…m 'master' to 'admin@example.org' to make it work with Stalwart >= 0.16
 * add instructions and container to create a key and certificate pair
   for the built-in IDM LDAP as we need it to listen on LDAPS for
   Stalwart, which has been disabled by default with
   #2880

 * add environment variables to point to the certificates

 * update services/groupware/DEVELOPERS.md with up-to-date instructions,
   using Stalwart 0.16
…ssions on the files or the opencloud container can't access them
 * fix the inline dockerfile for building the stalwart:cli container,
   cannot be based on scratch since we need a shell and wget, and it
   should thus retain the alpine container as its base

 * remove stdin_open and tty from the container attributes, not needed
…ements for using the primary email instead of the username
…ailable for 1.25 at the moment

 * retrofit from using self-referencing generics parameters that were
   introduced with Go 1.26, will re-enable when we upgrade to 1.26

 * re-introduce a 'ptr' func since we don't have the new 'new' func yet
   that comes with 1.26
…E_MOCK_DATA that controls whether the mock addressbook and mock contacts should be injected or not; default is false/disabled since it is not fully functional yet
 * by adding a predicate that can analyze a filter and inform the
   template methods whether the filter is applicable for that supplier
   or not

 * this fixed /groupware/accounts/.../addressbooks/.../contacts
 * implement supplier templating to allow addressbooks and contacts to
   come from various sources ("suppliers", thus) as opposed to solely
   from JMAP

 * add creating, deleting, updating, getting changes

 * add some search filter parameters, currently incomplete
…: in that case, don't use impersonation but the impersonation account directly
 * introduce configuration settings for TLS insecurity, tracing requests
   and responses

 * use more secure implementation when replacing JMAP placeholders in
   URL templates, path escaping them to avoid injection

 * add more efficient tracing of HTTP requests and responses between the
   Groupware backend and the JMAP server, with the possibility of
   specifying a maximum body size to trace, to avoid blowing up the logs

 * use more efficient string building for HTTP Authorization headers

 * change internal API to use streams (io.Reader) instead of reading
   JSON responses fully into memory before parsing them
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status:Needs-Review Type:Maintenance E.g. technical debt, packaging, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants