bachelor_thesis/30_Thesis/sections/50_design_and_implementation.tex

\chapter{Design and Implementation}
\label{ch:DesignImplementation}

This thesis requires a group configuration system with a recommender. In this thesis \emph{CAS Configurator Merlin} \cite{IndustrySpecificProduct2020} is extended. As this configurator does not allow group configuration, a modified version is used that has been developed by \citeauthor{raabKollaborativeProduktkonfigurationEchtzeit2019} \cite{raabKollaborativeProduktkonfigurationEchtzeit2019}. The base system and its extension are described in \autoref{sec:Foundations:BaseSystem}.

\autoref{fig:DesignImplementation:RecommenderForCollaborativeConfiguratorMerlin} shows how the system architecture after the recommender is added. For this thesis M.Collab and M.Collab-Customer have undergone some slight modifications to allow the displaying of recommendations and to query the recommender. The user interface is shown in \autoref{fig:DesignImplementation:RecommenderForCollaborativeConfiguratorMerlin}. It extends the user interface of M.Collab-Customer by adding a like and dislike button \todo[]{bisher waren deine Präferenzen dargestellt über Werte im Bereich [0;1], jetzt taucht nur like/dislike auf. Erkläärung, wie neutral dargestellt wird.
Das musst du entweder ändern oder die Inkonsistenz gut wegargumentieren.} and functionality to show other participants in the session.

\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{./figures/50_design_and_implementation/user_interface_prototype.png}
    \caption{The user interface of the prototype, seen by one user while configuring with three other people. The circles in the top left represent the users that are connected in the current sessions. The current configuration state is shown by the characteristics coloured in red. Recommendations are surrounded with a green border.}
    \label{fig:DesignImplementation:UserInterface}
\end{figure}


\section{Microservice}
\label{sec:DesignImplementation:Microservice}

An existing base system can be extended in different directions. As M.Collab already functions as middleman between M.Collab-Customer and M.Core, one solution would be to add recommender functionality into M.Collab-Customer. However, this approach would conflict with the single responsibility and separation of concern software design principles. It would change M.Collab from a system that works similar to a router to one that additionally handles  recommendations. M.Collab manages communication between M.Collab-Customer instances and between M.Core and M.Collab-Customer. Therefore this solution was discarded.
Another viable solution is adding the recommendation functionality into M.Core. The main benefit of this approach is that it allows the recommender to use systems that are used for solving constraint satisfaction problems to enhance recommendations. The disadvantage of this approach is however, that now the recommender is tightly coupled with the configurator. Not only does this mean that reproducing results in this thesis is only possible with access to a non open-source product but it also means that there is no possibility to use the recommender for other configuration solutions. Moreover no plugin API exists and therefore the extension would require a large effort. 
Due to these reasons the decision was made to create a new microservice, called M.Recommend which communicates with M.Collab, thereby being loosely coupled with other system components. Additionally, this approach allows to add functionality that takes  advantage of a constraint satisfaction problem solver by either communicating with one via an API or by integrating one directly into the recommender M.Recommend. Moreover, it also allows the use of different technologies instead of JavaScript and Node.js. Using a separate component also allows to scale the system differently as it is possible to use multiple recommender instances for one configurator instance or one recommender instance for multiple configurator instances. This is an advantage in the age of automatically scaling systems.

\begin{figure}
    \centering
    \includegraphics[width=0.6\textwidth]{./figures/50_design_and_implementation/MerlinCollabRecommender.pdf}
    \caption{The architecture of the extended recommender system based on \cite[Fig. 4.3]{raabKollaborativeProduktkonfigurationEchtzeit2019}'s configurator (see \autoref{fig:Foundations:CollaborativeConfiguratorMerlin}). The changes include the addition of the component \emph{M.Recommend} and the recommender database. Changes are marked in grey. Components that have a grey background are added and components with a partial grey background undergo changes.}
    \label{fig:DesignImplementation:RecommenderForCollaborativeConfiguratorMerlin}
\end{figure}

\section{API}
\label{sec:DesignImplementation:API}

The API used for the recommender is a REST api. REST allows the flexible addition of new endpoints thereby making the system easy to extend.
When a recommendation is supposed to be generated, M.Collab sends a HTTP-POST-request to M.Recommend (see \autoref{fig:DesignImplementation:SequenceDiagramRecommendation}). This request contains preferences and the current configuration. M.Recommend generates a recommendation based on the received data as a response. The recommended finished configuration is now sent from  M.Collab via broadcast over WebSocket to all M.Customer clients. 

\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{./figures/50_design_and_implementation/sequence_diagram_generating_recommendation.pdf}
    \caption{A sequence diagram showing the recommendation process.}
    \label{fig:DesignImplementation:SequenceDiagramRecommendation}
\end{figure}

The REST API looks as follows.

\begin{description}
    \item[/recommender/] is reachable via POST request and accepts a configuration state and preferences. Based on that a finished configuration is send back as response.
    \item[/config/] accepts GET and POST requests. Sending a get requests results in a list of all stored configurations being send back. POST is used for adding a finished configuration to the configuration database.
    \item[/product\_structure/] is reachable via GET and PUT. GET returns the current product structure and PUT is there to replace it.
\end{description}

The API is implemented with a minimal amount of functions and the recommender currently only supports one product at the same time. However, the architecture is built in a way that can be easily extended into supporting multiple products.
The general architecture adheres to the model view controller inspired architecture.
Data is stored and retrieved using a data access objects therefore the currently used simple TinyDB database backend can be switched to a different one easily.

\section{Database}
\label{sec:DesignImplementation:Database}

The choice among database systems has to be made between \emph{non-relational} and \emph{relational} databases. Relational databases are great at the four ACID (atomicity, consistency, isolation, durability) principles. Moreover if the data structures are not changing it provides a solid basis that keeps the data reliable. A non-relational database on the other hand is ideal for rapid agile development. Moreover it shines if data requirements are not entirely clear and if a large amount of unstructured  data has to be stored. Moreover non-relational databases allows the system to store the data in any kind of structure. This proves as an advantage as it allows to use the same data structure to be stored that also has to be fed out through the api. Therefore parsing methods for the API can be reused and changed upon changing requirements. Moreover the data used for the recommender is mostly not interconnected therefore a relational databases main strength the data structure does not really come into play here. Thus in this thesis a NoSQL database is used. 

\section{Scoring Functions}
\label{sec:DesignImplementation:ScroingFunctions}

Scoring functions are implemented in a modular fashion. There are different types of scoring functions. Some are meant for evaluating user preferences (and inherit from the class \emph{PreferenceScoringFunction}), others are meant for penalising changes in the current configuration state and therefore inheriting from  \emph{ConfigurationPenalty}. The last type of scoring function is a composite scoring function that consists of one or multiple scoring functions and combines their score into one. It uses the \emph{composite} design pattern. These functions allow ´the flexible combination of functions but moreover because of their underlying structure they can use similar components. For example preference scoring functions uses a pipeline to first convert preferences to a list of values. This is done using \emph{PreferenceToListFunction}. A list can be modified with a \emph{ListToListFunction}. This step is optional. The last mandatory step is to convert the list to a value using \emph{ListToValueFunction}. After this step the value can be modified further, for example with a threshold function or any other function. All these components allow the easy and quick creation of new scoring functions. A scoring function factory is used to assemble a scoring function based on input parameters. This gives flexibility for quickly implementing new types of scoring functions.


\section{Technologies}
\label{sec:DesignImplementation:Technologies}

M.Recommend is built in \emph{Python} \cite{PythonOrg} using \emph{FlaskRESTPlus} \cite{FlaskRESTPlus13Documentation}. \emph{FlaskRESTPlus} is an extension of \emph{Flask} \cite{FlaskDocumentation} that is meant to build a REST-Service and allows the generation of a web UI which documents the API. \emph{Flask} is a WSGI \cite{WhatWSGI} framework that is meant to easily get started with development while maintaining the ability to scale up to complex applications.

As Database system \emph{TinyDB} \cite{TinyDB15Documentation} with the extension \emph{tinyrecord} \cite{junEugeneeeoTinyrecord2020} is used. \emph{TinyDB} is a simple well tested NoSQL database framework that can store documents that are represented as dictionaries. The extension \emph{tinyrecord} adds atomic transaction support with a record-first then execute architecture.

As testing library for unittests \emph{pytest} \cite{PytestDocumentation} is used which is commonly used for python. It supports automatic test discovery.

For the evaluation of data and also for some operations with lists \emph{numpy} \cite{NumPy} is used. This library is optimised for fast scientific computing. 

Visualisation is done using \emph{Matplotlib} \cite{MatplotlibDocumentation} in combination with \emph{pandas} \cite{PandasPythonData}. \emph{Pandas} is a powerful framework for data analysis and manipulation tool which in the context of this thesis is used to load evaluation data.