diff --git a/src/01_management-summary/management-summary.tex b/src/01_management-summary/management-summary.tex index c1540eece015480a7209333022bfd1313d98193e..6e0bd4109b826a1c65a9311fb2852c8ad0070469 100644 --- a/src/01_management-summary/management-summary.tex +++ b/src/01_management-summary/management-summary.tex @@ -1,22 +1,67 @@ -\chapter*{Management Summary} +\chapter{Management Summary} -\instructions{ - Even though this is the first chapter of your document, it is typically the last one filled with content. The \textit{Management Summary} is a \textbf{brief and high-level summary} of your project. It should give any reader unfamiliar to the project an overview of the contents included later in the document. - - \bigskip - - A common structure is: - - \begin{itemize} - \item What is the problem we wanted to solve? - \item How did we solve the problem? - \item Does your solution solve the problem in a successful way? - \item Will there be consecutive projects based on your work? - \end{itemize} - - Diagrams and images work very well in this chapter, especially screenshots of your software. - - \bigskip + \section{Problem} + As described in the \nameref{sec:project-documentation/initial-project-description}, the SmartEating platform lacks an interactive chatbot. + This chatbot should be able to assist the user with various cooking related tasks. + Additionally, the chatbot should integrate a GPT model from OpenAI as well as an large language model hosted at OST. + + \section{Techniques} + As a first step, \hyperref[sec:project-plan/processes-meetings]{Scrum} was selected as a short-term project management framework to iteratively create a working, presentable product. + To ensure that the project would be completed within the given timeframe, \hyperref[sec:project-documentation/long-term-plan/phases]{RUP} was chosen as a long-term project management framework. + I have chose these project management frameworks because I already gathered some experience with them in previous projects. + + \section{Results} + The final solution successfully addresses the problem by delivering a robust, user-friendly chatbot. + The user can choose between the \hyperref[sec:product-documentation/architecture/c4-diagrams/code-diagram/openai]{OpenAI assistant}, which uses the OpenAI API and the \hyperref[sec:product-documentation/architecture/c4-diagrams/code-diagram/mixtral]{Mixtral assistant}, which uses the Mixtral model hosted at OST. + The chatbot was developed such that a new assistant can be easily integrated. + + \begin{figure}[H] + \centering + \noindent\makebox[\textwidth]{ + \begin{minipage}[H]{0.4\textwidth} + \begin{figure}[H] + \centering + \includegraphics[width=\linewidth]{screenshots/chatbot_assistant_selection.png} + \caption{Assistant selection} + \end{figure} + \end{minipage} + \begin{minipage}[H]{0.4\textwidth} + \begin{figure}[H] + \centering + \includegraphics[width=\linewidth]{screenshots/chatbot.png} + \caption{Assistant selection} + \end{figure} + \end{minipage} + \begin{minipage}[H]{0.4\textwidth} + \begin{figure}[H] + \centering + \includegraphics[width=\linewidth]{screenshots/chatbot_guidance.png} + \caption{Assistant selection} + \end{figure} + \end{minipage} + } + \end{figure} + + To help users make an informed decision about which assistant is best for them, the assistants are \hyperref[sec:appendix/assistant-comparison]{compared}. + + The chatbot is fully integrated into the SmartEating infrastructure. + + \begin{figure}[H] + \centering + \noindent\makebox[\textwidth]{ + \includegraphics[width=1.3\linewidth]{screenshots/smart_eating.png} + } + \caption{SmartEating platform with integrated chatbot} + \end{figure} - One final remark: a well written management summary is a good starting point for your \textbf{Project Presentation}, as you will address a similar audience there. -} + \section{Outlook} + During the project, several improvements were identified. + + The Mixtral assistant could be improved with further prompt engineering and providing a tool to search for the relevant recipe information. + The implementation of noise cancelling will allow users to use the chatbot even in noisy conditions. + + These improvements, as well as eliminating the \hyperref[sec:product-documentation/limitations]{current limitations}, will provide a better user experience. + + \section{Conclusion} + The \enquote{PanPal: A chef's chatbot} project integrates a robust and user-friendly chatbot into the existing SmartEating platform. + While there is still room for improvement to enhance the user experience, the resulting product provides the desired range of functionality. diff --git a/src/02_product-documentation/architecture/c4-diagrams.tex b/src/02_product-documentation/architecture/c4-diagrams.tex index 750a81bf5a603f682b947b6c5f7daed0c6476018..62a56d27830021b11fd677e7d8a1cd279ce2a37e 100644 --- a/src/02_product-documentation/architecture/c4-diagrams.tex +++ b/src/02_product-documentation/architecture/c4-diagrams.tex @@ -29,7 +29,7 @@ Given the absence of a open-source and freely accessible alternative to the OpenAI text-to-speech model, the OpenAI model is currently employed for text-to-speech, even in the event that another assistant, such as the Mixtral assistant, is selected. - \subsection{Code diagram}\label{sec:src/product-documentation/architecture/c4-diagrams/code-diagram} + \subsection{Code diagram}\label{sec:product-documentation/architecture/c4-diagrams/code-diagram} \begin{figure}[H] \centering \includegraphics[width=\linewidth]{diagrams/architecture/c4/high_level_class_diagram.png} @@ -38,7 +38,7 @@ Now let's have a closer look at the assistant implementations. - \subsubsection{OpenAI Assistant Code Diagram} + \subsubsection{OpenAI Assistant Code Diagram}\label{sec:product-documentation/architecture/c4-diagrams/code-diagram/openai} \begin{figure}[H] \centering \makebox[\textwidth]{ @@ -47,11 +47,15 @@ \caption{OpenAI Assistant Code Diagram} \end{figure} - \subsubsection{Mixtral Assistant Code Diagram} - \begin{figure}[H]\label{sec:product-documentation/architecture/c4-diagrams/code-diagram/mixtral} + The OpenAI assistant uses GPT-3.5 via the the OpenAI API. + + \subsubsection{Mixtral Assistant Code Diagram}\label{sec:product-documentation/architecture/c4-diagrams/code-diagram/mixtral} + \begin{figure}[H] \centering \makebox[\textwidth]{ \includegraphics[width=1.3\linewidth]{diagrams/architecture/c4/mixtral_class_diagram.png} } \caption{Mixtral Assistant Code Diagram} \end{figure} + + The Mixtral assistant uses the Mixtral model hosted at OST. diff --git a/src/02_product-documentation/architecture/project-architecture.tex b/src/02_product-documentation/architecture/project-architecture.tex index 934726507a39061fe590ce2aa370e289a172b7d6..d4529096972b45bf1ef4b4b23a156fceb82a93c3 100644 --- a/src/02_product-documentation/architecture/project-architecture.tex +++ b/src/02_product-documentation/architecture/project-architecture.tex @@ -24,7 +24,7 @@ The main.py sets up the API and handles the API calls. It mainly interacts with the \hyperref[sec:src/product-documentation/architecture/project-architecture/chatbot-project-structure/assistants-module-structure]{assistants}. - The \hyperref[sec:src/product-documentation/architecture/c4-diagrams/code-diagram]{code diagrams} provide an overview of these interactions. + The \hyperref[sec:product-documentation/architecture/c4-diagrams/code-diagram]{code diagrams} provide an overview of these interactions. The assistants module contains all of the logic associated with the assistants. The section on the \hyperref[sec:src/product-documentation/architecture/project-architecture/chatbot-project-structure/assistants-module-structure]{assistants structure} offers a more detailed analysis of the module's structure. diff --git a/src/02_product-documentation/limitations.tex b/src/02_product-documentation/limitations.tex index b11aae5a4deccf736020fff145bdc2f898d162f2..4052ffdacbde26c69a89e3aeb92c4bf8e627b20e 100644 --- a/src/02_product-documentation/limitations.tex +++ b/src/02_product-documentation/limitations.tex @@ -1,4 +1,4 @@ -\chapter{Limitations} +\chapter{Limitations}\label{sec:product-documentation/limitations} This chapter describes the limitations of the SmartEating chatbot. \section{Audio limitations} @@ -16,3 +16,8 @@ \section{Language limitations} The chatbot currently supports only English for both text messages and audio messages. Therefore, users should only interact with the chatbot in English. + + \section{Assistant limitations} + While the OpenAI assistant completes all tasks without any problems, the Mixtral assistant struggles with some tasks. + Specifically, the Mixtral assistant hallucinates ingredient quantities and does not provide step-by-step instructions, even when asked. + More information about this can be found in the \hyperref[sec:appendix/testing-protocols/llm-behavior-tests]{LLM behavior tests}. diff --git a/src/03_project-documentation/initial-project-description.tex b/src/03_project-documentation/initial-project-description.tex index 37988e27f7c6dadc7eabdac44eb994de58077815..d33dc26a9c0351a7aeeb587751ab099851b19595 100644 --- a/src/03_project-documentation/initial-project-description.tex +++ b/src/03_project-documentation/initial-project-description.tex @@ -1,4 +1,4 @@ -\chapter{Initial Project Description} +\chapter{Initial Project Description}\label{sec:project-documentation/initial-project-description} \section{Supervisor} Mitra Purandare (\href{mailto:mitra.purandare@ost.ch}{mitra.purandare@ost.ch}) diff --git a/src/03_project-documentation/project-plan/long-term-plan.tex b/src/03_project-documentation/project-plan/long-term-plan.tex index e50cd77b3d554474fc3c38c83781d95fa5bea92f..2ba6195a56a8bd3321fe16afed072f82a76b622b 100644 --- a/src/03_project-documentation/project-plan/long-term-plan.tex +++ b/src/03_project-documentation/project-plan/long-term-plan.tex @@ -34,6 +34,7 @@ \item All in-house models as assistants \item Saving chat over sessions \item Receipe suggestions + \item Noise cancelling for audio \end{itemize} \subsection{Epics}\label{sec:project-documentation/long-term-plan/epics} diff --git a/src/resources/screenshots/chatbot.png b/src/resources/screenshots/chatbot.png new file mode 100644 index 0000000000000000000000000000000000000000..b0e4fc9047b51ca6dd97f2e8e50eddbd9053f60e Binary files /dev/null and b/src/resources/screenshots/chatbot.png differ diff --git a/src/resources/screenshots/chatbot_assistant_selection.png b/src/resources/screenshots/chatbot_assistant_selection.png new file mode 100644 index 0000000000000000000000000000000000000000..5e57823c2473e88f02776fa95fa1653a81c8fae5 Binary files /dev/null and b/src/resources/screenshots/chatbot_assistant_selection.png differ diff --git a/src/resources/screenshots/chatbot_guidance.png b/src/resources/screenshots/chatbot_guidance.png new file mode 100644 index 0000000000000000000000000000000000000000..7eb09022246aa78c4aefa197618f55858b27b086 Binary files /dev/null and b/src/resources/screenshots/chatbot_guidance.png differ diff --git a/src/resources/screenshots/smart_eating.png b/src/resources/screenshots/smart_eating.png new file mode 100644 index 0000000000000000000000000000000000000000..d6449a1279029b93513ec93916ec40e2790f35d7 Binary files /dev/null and b/src/resources/screenshots/smart_eating.png differ diff --git a/src/title-page.tex b/src/title-page.tex index f387479f893346e9286582217744ee0d7752c9c3..e0c5de28ae5ec7512147fbab0741dc8da032e10a 100644 --- a/src/title-page.tex +++ b/src/title-page.tex @@ -34,7 +34,7 @@ \vfill - School of Computer Science\\ + Department of Computer Science\\ OST Eastern Switzerland University of Applied Sciences \end{center}