Rework infrastructure documentation and fix a few issues

Benjamin Rühl
1 parent 663586ee
Showing 6 changed files with 91 additions and 41 deletions
README.md
docu/apiai.md
docu/infrastructure.md
docu/jboss.md
docu/persistence.md
docu/project_documentation.md
@@ -26,14 +26,15 @@ BeuthBot is a master project of the Beuth University of Applied Sciences Berlin.
 - Oracle Java 8
 - Gradle Build - build system
-- Postgres SQL - relational database
+- PostgreSQL - relational database
+- Hibernate - object relational mapping
 - Adminer - database administration
 - WildFly 10
 - Docker
 <!-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -->
-## Firtst steps
+## First steps
 The following five steps clarify what is necessary to set up the BeuthBot for the first time. Most of the steps are also useful during development to ultimately test the changes in the application.
@@ -108,7 +109,23 @@ The following command can be used to stop the running application:
 ## Project documention
-In addition to this general documentation, the directory `docu/` also contains the [project_documentation](docu/project_documentation.md). This section also discuss the individual modules of the application.
+In addition to this general documentation, the directory `docu/` also contains several sections about the project documentation itself.
+
+General
+
+- [infrastructure](docu/infrastructure.md) - The individual modules of the application
+- [persistence](docu/persistence.md) - DAOs, Entities, Hibernate, PostgreSQL
+
+Individual modules
+
+- [api.ai](docu/apiai.md) - Usage of API.ai natural language processing
+- [attachment store](docu/attachmentstore.md) - Storing files that come with chat messages
+- [bing speech](docu/bingspeechapi.md) - Usage of bing speech API
+- [canteen parser](docu/canteenParser.md) - Extracting data from the canteen HTML web page
+- [drools](docu/drools.md) - Manage business rules defining the chat bot's behavior
+- [facebook](docu/facebook.md) - Facebook adapter, utils and Facebook app
+- [rasa](docu/rasa.md) - Bringing data from API.ai to Rasa
+- [telegram](docu/telegram.md) - Telegram bot API and Telegram adapter
 <!-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -->
@@ -195,7 +212,7 @@ NOTE: Localtunnel tends to crash every now and then, displaying an error message
 ## Debugging
-At the moment, the JBoss server is always started with open debugging ports. To attach to the process using IntelliJ, a remote debug configuration has to be created. To do so, perform the following steps:
+At the moment, the JBoss server is always started with open debugging ports. To attach to the running docker container process using IntelliJ, a remote debug configuration has to be created. To do so, perform the following steps:
 - Go to `Run` -> `Edit Configurations...`
 - Add a new configuration using the button in the top left corner of the dialog
 # api.ai API
+**IMPORTANT NOTE: API.ai by Google has become Dialogflow. That's why some information on this page might be out of date.**
+
 <!-- MarkdownTOC -->
 - [api.ai Setup](#api.ai-setup)
+# Infrastructure of BHT-Chatbot Project
+
+The whole project is built using gradle and divided into a couple of sub projects (cf. [Sub projects section](#subprojects)).
+Each module is loosely connected through a Java Message Service. The application is running on a JBoss Wildfly 10 inside of a docker container. Another docker container is used for the Rasa backend, one for the PostgreSQL database and one for the Adminer database management tool.
+
+- see [docker compose file](../docker/docker-compose.yml)
+
+The productive project is represented by a separate Git repository in absent of a continuous integration server.
+Pushing into the master branch of this repository should automatically trigger a rebuild of the productive environment.
+- confer [post-receive](../scripts/post-receive) - Git hook for auto deploying the application
+
+## Sub projects
+
+_**DISCLAIMER:** Please note that the following information is documented by the second project team who felt that a deeper look into the project structure should be a requirement for teams working on the project. This means that information is written 'as understood'. The original designers of the system had great knowledge of their architecture but sadly forgot to share it._
+
+The different modules of the application are deployed as individual services where no service knows whether the other services exist or not. The Java Massage Service is a bus like system where a service can send a message with a key and interested services can wait for messages with a certain key.
+An exception to this is the `common` module which is referenced by every other module. To lesser the dependencies here, the `common` module contains only interfaces and enums. The implementations to these interfaces lie in the `global` module which is unknown by the other modules.
+
+The `mainBot` module contains the core chat bot logic defining the specific use cases the bot supports.
+
+To read more about modules in Wildfly 10, see [JBoss configuration and usage](jboss.md).
+
+To read the story behind a very nasty issue with global entity classes, see [Why is the persistence layer located in `common` instead of `global`?](persistence.md#Why-is-the-persistence-layer-located-in-`common`-instead-of-`global`?)
+
+### MainBot
+
+- [Canteen Parser](canteenParser.md) - web crawler for collecting data of the beuth university canteen
+- Common - holding common classes used by all other subprojects
+- [Drools](drools.md) - Business Rules Management System used to generate the right answer
+- Global - global available services
+
+### Natural Language Processing
+
+- [ApiAi](apiai.md) - simple RESTEasy client application calling googles Api.ai API
+- [Rasa](rasa.md) - simple RESTEasy client application calling the rasa backend rest API
+
+### Messenger
+
+- [Facebook](facebook.md) - Facebook Messenger connector
+- [Telegram](telegram.md) - Telegram Messenger connector
+
+### Text <-> Speech Processing
+
+- [Bing Speech](bingspeechapi.md) - REST client for Microsofts Bing Speech API
 \ No newline at end of file
+# JBoss configuration and usage
+
+The intent of this file is not to provide a full documentation about JBoss, EJBs or dependency injection but to give some hints that might be useful when working on the project.
+
+## Main configuration file
+
+A JBoss application server is configured mostly using XML files. The most important one is the `standalone.xml` which is located in `docker/wildfly/standalone.xaml`.
+
+## Building and deploying the services
+
+When building with gradle, the modules result in seperate `.war` archives which are copied into the JBoss docker container. JBoss should then reload them automatically.
 \ No newline at end of file
@@ -42,7 +42,7 @@ public interface AppUserDAO extends GenericDAO&lt;AppUser, Long&gt; {
 }
 ```
-# How can a DAO instance be used?
+## How can a DAO instance be used?
 - using dependency injection in a class that is managed by JBoss
@@ -51,7 +51,7 @@ public interface AppUserDAO extends GenericDAO&lt;AppUser, Long&gt; {
 private AppUserDAO userDAO;
 ```
-# How to create an entity?
+## How to create an entity?
 - currently a DAO is responsible for creating instances of its context entity, e.g. `AppUser newUser = userDAO.createUser()`
 - direct creation of an entity like `AppUser newUser = new AppUser()` were not possible at first, but could be used now, too
@@ -66,7 +66,7 @@ public interface AppUserDAO extends GenericDAO&lt;AppUser, Long&gt; {
 - the entity can then be modified and updated in the database using the DAO instance
-# AppUsers have generic attributes. How can they be used?
+## AppUsers have generic attributes. How can they be used?
 - the **Entity-Attribute-Value model (EAV)** is a common pattern in relational database design which aims at structuring a database schema in rows instead of columns; please do some research to really understand it
 - the pattern is implemented and used in the `AppUser` entity to give developers the ability to define attributes at runtime
@@ -88,11 +88,11 @@ userDAO.saveOrUpdate(u);
 - instead of this, the EAV structure can be build manually, too
 - please note that complex data types and lists are not supported using this syntax at the moment; to add support, the custom serializers and deserializers found in `common/persistence` must be extended
-# How does the implementation of the EAV pattern work internally for the AppUser entity?
+## How does the implementation of the EAV pattern work internally for the AppUser entity?
 ![Domain model as UML class diagram](img/persistence/domain-model.png)
-# How to retrieve the generic attributes of an AppUser?
+## How to retrieve the generic attributes of an AppUser?
 - the following snippet shows the retrieval using the convenience method
 - instead of this, the EAV structure can be queried an walked manually, too
@@ -105,7 +105,7 @@ boolean testValueBool = u.getProperty(&quot;test_property_bool&quot;, Boolean.class);
 Double testValueDouble = u.getProperty("test_property_double", Double.class);
 ```
-# How does the convenience method for accessing the `GenericEntity` hierarchy actually work?
+## How does the convenience method for accessing the `GenericEntity` hierarchy actually work?
 A trivial implementation providing access to the structure with getters and setters is provided, but it gives the responsibility of traversing this structure to the using component. To simplify this for developers, a mechanic using custom JSON serializers and deserializers has been implemented.
 When setting a generic attribute, the given value is serialized to JSON using a common serializer. The value as JSON is then given to the custom deserializers which can create a GenericEntity structure from this JSON. 
@@ -174,4 +174,10 @@ The custom serializers instead have something like the following output:
 }
 ```
-This result can be used as input for a common JSON deserializer which can create an instance of a desired type from it as long as the type has exactly the same attributes as the structure has in JSON.
 \ No newline at end of file
+This result can be used as input for a common JSON deserializer which can create an instance of a desired type from it as long as the type has exactly the same attributes as the structure has in JSON.
+
+## Why is the persistence layer located in `common` instead of `global`?
+
+Short story that might be useful to read: When implementing the persistence layer, the approach was to define entity interfaces and DAO interfaces in `common` and seperate implementations in `global`. From outside of the `global` module a developer should only be able to interact with the entities through their interfaces. Although this is the clean way which even compiled without problems, the entity instances could not be used all the time. In some cases a `ClassCastException` arose when trying to cast an entity instance to its interface. This should work fine normally. Research showed that **each module in Wildfly 10 has its own class loader**. That is why an instance created in a module which is not directly referenced can lead to problems. This can even happen when casting an object to its own class because the instance and the class have been loaded by different class loaders that cannot resolve types loaded by the other loader.
+
+Because nobody was able to deal with this problem correctly, we decided to move the whole persistence layer implementation to `common` although this is clearly a design flaw.
 \ No newline at end of file
-# BHT-Chatbot Project
-The hole project is build by gradle and divided into a couple of subprojects (cf. [Subprojects section](#subprojects) ).
-Each module is loosely connected through a Java Message Service. The application is running on a Jboss Wildfly
-inside of a docker container. Another docker container is used for the Rasa backend.
-
-## Infrastructure
-- see [docker compose file](../docker/docker-compose.yml)
-
-The productive project is represented by a separate Git repository in absent of a continuous integration server.
-Pushing into this repository will automatically trigger a rebuild of the productive environment.
-- confer [post-receive](../scripts/post-receive) - Git hook for auto deploying the application
-
-### Subprojects
-
-#### MainBot
-- [Canteen Parser](canteenParser.md) - web crawler for collecting data of the beuth university canteen
-- Common - holding common classes used by all other subprojects
-- [Drools](drools.md) - Business Rules Management System used to generate the right answer
-- Global - global available services
-
-#### Natural Language Processing
-- [ApiAi](apiai.md) - simple RESTEasy client application calling googles Api.ai API
-- [Rasa](rasa.md) - simple RESTEasy client application calling the rasa backend rest API
-
-#### Messenger
-- [Facebook](facebook.md) - Facebook Messenger connector
-- [Telegram](telegram.md) - Telegram Messenger connector
-
-#### Text <-> Speech Processing
-- [Bing Speech](binspeechapi.md) - REST client for Microsofts Bing Speech API
 \ No newline at end of file