[buteo-syncfw] Rework buteo plugin API for out-of-process-plugins API…

… and add buteo-oopp-runner. JB#52916 The current out-of-process plugin API requires plugins to compile in various .cpp files that are bundled in the devel package headers. This means all plugins compile in the same source files when they could just link in some plugin header files instead, and be launched by a plugin helper binary. Move the plugin helper files into oopp-runner and add a buteo-oopp-runner binary that launches out-of-process plugins. This breaks source compatibility for plugins as they now need to add a Buteo::SyncPluginLoader or Buteo::StoragePluginLoader subclass that creates and destroys the plugin class as necessary. DummyClient and DummyServer in unittests/dummyplugins have been updated to reflect the changes. The template generation tools have been removed as they are no longer used and were also generating code for outdated API. Plugins are now determined to be out-of-process purely based on whether they are installed in /usr/lib/buteo-plugins-qt5 (in-process) or /usr/lib/buteo-plugins-qt5/oopp (out-of-process).
sailfishos · Apr 2, 2021 · 36e1cf9 · 36e1cf9
1 parent 8e837d2
commit 36e1cf9
Show file tree

Hide file tree

Showing 49 changed files with 838 additions and 3,613 deletions.
diff --git a/buteo-sync.pro b/buteo-sync.pro
@@ -3,11 +3,12 @@ TEMPLATE = subdirs
 SUBDIRS += \
     libbuteosyncfw \
     msyncd \
+    oopp-runner \
     unittests \
-    tools \
     doc
 
 msyncd.depends = libbuteosyncfw
+oopp-runner.depends = libbuteosyncfw
 unittests.depends = libbuteosyncfw msyncd
 
 coverage.CONFIG += recursive

diff --git a/doc/src/Outofprocesspluginclasses.png b/doc/src/Outofprocesspluginclasses.png
diff --git a/doc/src/introduction.dox b/doc/src/introduction.dox
@@ -590,36 +590,24 @@ of the API
 </td></tr></table>
 
 \section support-oop-plugins Support for out of process plugins
+
 Buteo syncfw version 0.1.0 supported only dynamic link library plugins which the
-framework loads into the same process memory as msyncd. This archicture has a
+framework loads into the same process memory as msyncd. This architecture has a
 problem that if any one of the plugin misbehaves (crashes, for example), msyncd
 would also crash and there is a probability that it would never recover. To avoid
 such situations, an out of process plugin architecture was devised and implemented.
 In this architecture, each of the plugins would be running as separate processes
 and msyncd process would interact with each of the processes to handle the sync
 life cycle and operations.
 
-This new architecture is devised with the following absolute requirements:
-
-- None of the plugin code should change. This is to ensure backward compatibility
-of all existing plugins
-- The flow of msyncd should not change w.r.t the new plugins, ie., the flow remains
-the same for both DLL plugins as well as process plugins
-- Support for DLL plugins still continues and they can co-exist along with process
-plugins 
-
-Keeping the above requirements in context, the following architecture is devised,
-the class diagram of which is depicted below. 
+From Buteo syncfw version 0.10.0, the plugin architecture uses Qt Plugins
+(https://doc.qt.io/qt-5/plugins-howto.html). Plugin files installed in
+/usr/lib/buteo-plugins-qt5 are launched in-process; plugin files installed in
+/usr/lib/buteo-plugins-qt5/oopp are launched out-of-process using the
+buteo-oopp-runner binary.
 
-</p><p>\image html Outofprocesspluginclasses.png
-
-The classes in blue belong to the framework, while the classes in pink to the plugin.
-Two concrete classes, OOPClientPlugin and OOPServerPlugin (that inherit from ClientPlugin
-and ServerPlugin respectively) are created which act as the interface between the
-plugin and the framework. These classes will be responsible for performing any
-back-forth conversion of the data transferred. These classes also ensure that the
-interface between msyncd and the plugins remain the same. In dll based plugin,
-the .so file is opened using dlsym and the binary is loaded into memory and then
+Plugins should publicly inherit from Buteo::ClientPlugin or Buteo::ServerPlugin.
+In dll based plugins, the .so file is opened using dlsym and the binary is loaded into memory and then
 the ClientPlugin object is created. In this mechanism, the plugin process is started
 and then the OOPClientPlugin object is created, which then talks to the process
 plugin over d-bus.
@@ -644,8 +632,5 @@ PluginServiceObj and registers it as a dbus service.
 All signals emitted by the plugin (and msyncd) are automatically relayed by Qt dbus.
 The life cycle of a process plugin is similar to that of a dll plugin.
 
-To convert an existing dll plugin to a process plugin, only a few settings need
-to be added. Described here \ref oop-plugins
-
 </div>
 */
diff --git a/doc/src/plugin-guide.dox b/doc/src/plugin-guide.dox
@@ -19,102 +19,9 @@ databases, like Contacts, File system, Calendar etc. These plugins can then be
 used across multiple protocols like Dropbox, Google services, Caldav etc.
 
 \section create-plugin Creating a plugin
-The buteo-syncfw comes with a tools package to create template plugin code that
-will help the developer to quickly create plugins
 
-The tool uses Cheetah (www.cheetahtemplate.org/‎) template library to generate the
-templates. One can install cheetah in Ubuntu using command: 'sudo apt-get install
-python-cheetah'. Python 2.7 or greater is required 
-
-\subsection generate-plugin-template-code Instructions for generating a plugin
-boiler plate code.
-The configuration files corresponding to a client/server/storage have to be
-filled-in with the appropriate information. Each of the fields in the config files
-are marked with MANDATORY/OPTIONAL. Info about the fields is provided in the config
-files
-
-To generate client plugin, run the command:
-
-\code ./gen_template.py -c client-plugin.cfg -d <output dir> \endcode
-
-To generate server plugin, run the command:
-
-\code ./gen_template.py -c server-plugin.cfg -d <output dir> \endcode
-
-To generate storage plugin, run the command:
-
-\code ./gen_template.py -c storage-plugin.cfg -d <output dir>  \endcode
-
-
-\section oop-plugins Writing Out of process plugins
-Buteo syncfw version 0.1.0 supported only dynamic link library plugins which the
-framework loads into the same process memory as msyncd. This archicture has a
-problem that if any one of the plugin misbehaves (crashes, for example), msyncd
-would also crash and there is a probability that it would not recover. To avoid
-such situations, an out of process plugin architecture was devised and implemented.
-In this architecture, each of the plugins would be running as separate processes
-and msyncd process would interact with each of the processes to handle the sync
-life cycle and operations.
-
-In the new architecture, there is no need to modify any of the existing plugin code.
-The only change that needs to be done is in the .pro files of each of the plugins
-
-The following few lines have to be added to the .pro file to convert a .so plugin
-to a binary executable plugin
-
-\code
-
-TEMPLATE = app
-QT += dbus
-target.path = $$[QT_INSTALL_LIBS]/buteo-plugins-qt5/oopp
-
-DEFINES += "CLASSNAME=MyPluginClassname"
-DEFINES += CLASSNAME_H=\\\"MyPluginClassname.h\\\"
-INCLUDE_DIR = $$system(pkg-config --cflags buteosyncfw5|cut -f2 -d'I')
-
-HEADERS += $$INCLUDE_DIR/ButeoPluginIfAdaptor.h   \
-           $$INCLUDE_DIR/PluginCbImpl.h           \
-           $$INCLUDE_DIR/PluginServiceObj.h
-
-SOURCES += $$INCLUDE_DIR/ButeoPluginIfAdaptor.cpp \
-           $$INCLUDE_DIR/PluginCbImpl.cpp         \
-           $$INCLUDE_DIR/PluginServiceObj.cpp     \
-           $$INCLUDE_DIR/plugin_main.cpp
-
-\endcode
-
-In the above, replace "MyPluginClassname" with the name of your plugin class
-and the name of the header
-
-If the plugin is a client plugin, add the following DEFINES, else <b>do not</b> add
-
-\code
-DEFINES += CLIENT_PLUGIN
-\endcode
-
-Also, note that your plugin should have public inheritance from Buteo::ClientPlugin 
-
-If you want your existing plugin dll configuration to exist with the out of process plugin configuration, the good way is to add another configuration to your .pro file. The config would look like:
-
-\code
-PLUGIN_DLL {
-...
-# DLL configuration (the same as your existing configuration)
-...
-}
-
-PLUGIN_EXE {
-...
-# OOP plugin config
-...
-}
-\endcode
-
-To generate the code for a specific configuration, you can run qmake like:
-
-\code
-qmake myplugin.pro CONFIG+=PLUGIN_EXE
-\endcode
+See the DummyClient and DummyServer classes in unittests/dummyplugins for
+plugin examples.
 
 \section libsyncml Device sync with libsyncml
 libsyncml is a desktop based sync engine that support sync using SyncML

diff --git a/libbuteosyncfw/libbuteosyncfw.pro b/libbuteosyncfw/libbuteosyncfw.pro
@@ -33,9 +33,12 @@ HEADERS += common/Logger.h \
            pluginmgr/PluginManager.h \
            pluginmgr/ServerPlugin.h \
            pluginmgr/StorageChangeNotifierPlugin.h \
+           pluginmgr/StorageChangeNotifierPluginLoader.h \
            pluginmgr/StorageItem.h \
            pluginmgr/StoragePlugin.h \
+           pluginmgr/StoragePluginLoader.h \
            pluginmgr/SyncPluginBase.h \
+           pluginmgr/SyncPluginLoader.h \
            profile/BtHelper.h \
            profile/Profile.h \
            profile/Profile_p.h \
@@ -66,6 +69,7 @@ SOURCES += common/Logger.cpp \
            pluginmgr/StorageItem.cpp \
            pluginmgr/StoragePlugin.cpp \
            pluginmgr/SyncPluginBase.cpp \
+           pluginmgr/SyncPluginLoader.cpp \
            profile/BtHelper.cpp \
            profile/Profile.cpp \
            profile/ProfileFactory.cpp \
@@ -102,12 +106,6 @@ QMAKE_CLEAN += lib$${TARGET}.prl pkgconfig/*
 # install
 target.path = $$[QT_INSTALL_LIBS]
 headers.path = /usr/include/buteosyncfw5/
-sources.path = $$headers.path
-
-sources.files = pluginmgr/plugin_main.cpp \
-                pluginmgr/PluginServiceObj.cpp \
-                pluginmgr/ButeoPluginIfaceAdaptor.cpp \
-                pluginmgr/PluginCbImpl.cpp
 
 headers.files = common/Logger.h \
            common/LogMacros.h \
@@ -123,13 +121,13 @@ headers.files = common/Logger.h \
            pluginmgr/PluginManager.h \
            pluginmgr/ServerPlugin.h \
            pluginmgr/StorageChangeNotifierPlugin.h \
+           pluginmgr/StorageChangeNotifierPluginLoader.h \
            pluginmgr/StorageItem.h \
            pluginmgr/StoragePlugin.h \
+           pluginmgr/StoragePluginLoader.h \
            pluginmgr/SyncPluginBase.h \
-           pluginmgr/PluginServiceObj.h \
-           pluginmgr/ButeoPluginIfaceAdaptor.h \
+           pluginmgr/SyncPluginLoader.h \
            pluginmgr/ButeoPluginIface.h \
-           pluginmgr/PluginCbImpl.h \
            profile/BtHelper.h \
            profile/Profile.h \
            profile/Profile_p.h \
@@ -145,7 +143,7 @@ headers.files = common/Logger.h \
            profile/SyncSchedule_p.h \
            profile/TargetResults.h
 
-INSTALLS += target headers sources
+INSTALLS += target headers
 
 QMAKE_PKGCONFIG_DESTDIR = pkgconfig
 QMAKE_PKGCONFIG_LIBDIR  = $$target.path