13hannes11/bachelor_thesis
1
0
Fork 0
mirror of https://github.com/13hannes11/bachelor_thesis.git synced 2024-09-04 01:11:00 +02:00
Code Issues Projects Releases Wiki Activity

add hypotheses to threshold center in findings

Browse Source
This commit is contained in:
hannes.kuchelmeister
2020-03-30 12:34:21 +02:00
parent c1c34a1670
commit 939aab33a3
1 changed files with 4 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
30_Thesis/sections/60_evaluation.tex
View File

@@ -200,10 +200,6 @@ Understanding data is made easier by first posing hypotheses. This section gives
To get an understanding of the data all parameters except the $tc$ will be fixed. The preference aggregation strategy looked at is multiplication. The configuration database is used with all possible solutions (which is 148 in total). This results in a bigger visible effect as the recommender has access to all possible configurations. \autoref{fig:Evaluation:tcChange} shows the satisfaction change based on choice of $tc$. Of note is that the maxima of satisfaction change precedes the minima of dissatisfaction change for all group types. Maxima and minima occur at different tc values depending on the group type. Heterogeneous groups peek earliest while homogenous groups only show a peek towards the maximum $tc$ value. Changes in dissatisfaction are minimal even with $tc$ close to its maximum value. \autoref{fig:Evaluation:tcCount} shows the amount of group members satisfied and dissatisfied with a decision. The number of satisfied people decreases with an increasing $tc$ and its downward movement accelerates. The dissatisfaction curve shows a similar trend but in contrast here the number of dissatisfied group members increases with and increase in $tc$. The curve accelerates its growth analogues to the acceleration of the satisfaction curve. The behaviour of heterogeneous groups and random groups is similar but the curve for heterogeneous groups show less happiness for a given tc and more unhappiness. Also both curves have a negative satisfaction change when $tc$ reaches a certain height. Homogeneous groups only have happy group members for most $tc$ values but they decrease rapidly for values greater $85$. Dissatisfied group members are at zero for the whole value range of $tc$ except a very slight upward tick at the end that is barely noticeable. To get an understanding of the data all parameters except the $tc$ will be fixed. The preference aggregation strategy looked at is multiplication. The configuration database is used with all possible solutions (which is 148 in total). This results in a bigger visible effect as the recommender has access to all possible configurations. \autoref{fig:Evaluation:tcChange} shows the satisfaction change based on choice of $tc$. Of note is that the maxima of satisfaction change precedes the minima of dissatisfaction change for all group types. Maxima and minima occur at different tc values depending on the group type. Heterogeneous groups peek earliest while homogenous groups only show a peek towards the maximum $tc$ value. Changes in dissatisfaction are minimal even with $tc$ close to its maximum value. \autoref{fig:Evaluation:tcCount} shows the amount of group members satisfied and dissatisfied with a decision. The number of satisfied people decreases with an increasing $tc$ and its downward movement accelerates. The dissatisfaction curve shows a similar trend but in contrast here the number of dissatisfied group members increases with and increase in $tc$. The curve accelerates its growth analogues to the acceleration of the satisfaction curve. The behaviour of heterogeneous groups and random groups is similar but the curve for heterogeneous groups show less happiness for a given tc and more unhappiness. Also both curves have a negative satisfaction change when $tc$ reaches a certain height. Homogeneous groups only have happy group members for most $tc$ values but they decrease rapidly for values greater $85$. Dissatisfied group members are at zero for the whole value range of $tc$ except a very slight upward tick at the end that is barely noticeable.
\todo[inline]{take reference to hypothesises that are regarding tc}
During a group decision it is better to make one less person dissatisfied opposed to one more person satisfied. Therefore, this thesis uses $tc$ values that are closer to minima of unhappiness reduction than to the maxima of satisfaction change. The minima for heterogeneous is at $tc = 70$ therefore this is the chosen value for the evaluation of other aspects. For random groups the minima of dissatisfaction change can be found at $tc = 85$ which is the value used for all following analysis of random groups. For homogenous group dissatisfaction change is sinking until the highest value of $tc$. Because of that $tc = 94$ is used for analysis.
\begin{figure} \begin{figure}
\centering \centering
\includegraphics[width=1\textwidth]{./figures/60_evaluation/tc_change__multi__db-size-148.pdf} \includegraphics[width=1\textwidth]{./figures/60_evaluation/tc_change__multi__db-size-148.pdf}
@@ -218,13 +214,13 @@ During a group decision it is better to make one less person dissatisfied oppose
\label{fig:Evaluation:tcCount} \label{fig:Evaluation:tcCount}
\end{figure} \end{figure}
\subsection{Analysing Data} \hyporef{hyp:Evaluation:MaximumMinimum} states that the highest satisfaction change is expected at places where the overall satisfaction with the dictator's decision is one. However the data shows a slightly different result. This hypothesis does not hold true. When looking at the data we see peeks in satisfaction change when $2.81, 2.51, 3$ (heterogeneous, random, homogenous). Therefore the expectation does not hold up. Moreover, valleys for dissatisfaction change are also not at the expected value of \textit{two}. They are instead at $1.19, 1.49, 0.04$ (heterogeneous, random, homogenous). Here the valleys are lower than expected. However data from homogenous groups seems cut of therefore it is not possible to say if there could be a potentially bigger decrease if the solution space is bigger.
In this section results for heterogeneous groups, random groups and homogenous groups, based on the forest use case, are shown. \autoref{fig:Evaluation:HeterogenousGroupIncrease} and \autoref{fig:Evaluation:HeterogenousGroupTotal} show results for heterogeneous groups. \autoref{fig:Evaluation:RandomGroupIncrease}, \autoref{fig:Evaluation:RandomGroupTotal} shows the results for random groups and \autoref{fig:Evaluation:HomogenousGroupIncrease}, \autoref{fig:Evaluation:HomogenousGroupTotal} show the results for homogenous groups. The predicted trend that a higher $tc$ results in a lower satisfaction and a higher dissatisfaction, with the dictator's decision, as predicted by \hyporef{hyp:Evaluation:HigherTcLessSatisfied} can be clearly seen in \autoref{fig:Evaluation:tcCount} and has been described in this section already.
The first thing that is noticed when analysing the data is that with homogenous groups the recommender does not have any benefit to an individual choosing based on their own preferences. This is most likely due to all individuals being already satisfied with the individual decisions. This is an effect that was noticed even when a higher $smd$ was chosen. Here we notice that the effect of not having many configurations in the store does decrease hapiness by a large amount. \hyporef{hyp:Evaluation:OnlyOneSatisfied} predicts that the satisfaction with the individual decision eventually reaches one and that no one is satisfied with the group recommender decision. This means the satisfaction change should reach minus one. \autoref{fig:Evaluation:tcCount} shows a downward trend that come close to one for heterogeneous and random groups. Also homogenous groups see a big drop but this drop does not reach one. Nonetheless, the steep drop suggests that the hypothesis holds in regards to reaching only one person satisfied with the individual decision when using quantiles that do not have to be integers. Also, satisfaction change in heterogeneous groups reaches close to minus one but this value is neither reached by random groups, nor by homogenous groups. The hypothesis therefore should not be seen as confirmed in that regard and further investigation is needed.
When looking at results for random and heterogeneous groups the satisfaction level with an individual decision is much lower than individual decisions in homogenous groups. This finding is expected as random and homogenous groups are more diverse therefore opposing interest will be visible in these. During a group decision it is better to make one less person dissatisfied opposed to one more person satisfied. Therefore, this thesis uses $tc$ values that are closer to minima of unhappiness reduction than to the maxima of satisfaction change. The minima for heterogeneous is at $tc = 70$ therefore this is the chosen value for the evaluation of other aspects. For random groups the minima of dissatisfaction change can be found at $tc = 85$ which is the value used for all following analysis of random groups. For homogenous group dissatisfaction change is sinking until the highest value of $tc$. Because of that $tc = 94$ is used for analysis.
All scoring functions are similarly good in decreasing dissatisfaction. However the results differ when looking at satisfaction, Here least misery performs abysmal compared to the other scoring functions. In \autoref{fig:Evaluation:HeterogenousGroupIncrease} it results even in a hapiness reduction whereby multiplication and best average increase it. Overall multiplication seems to perform the best in most scenarios. This confirms findings in expirments with real people as described by \citeauthor{Masthoff2015} \cite[p. 755f]{Masthoff2015}. All scoring functions are similarly good in decreasing dissatisfaction. However the results differ when looking at satisfaction, Here least misery performs abysmal compared to the other scoring functions. In \autoref{fig:Evaluation:HeterogenousGroupIncrease} it results even in a hapiness reduction whereby multiplication and best average increase it. Overall multiplication seems to perform the best in most scenarios. This confirms findings in expirments with real people as described by \citeauthor{Masthoff2015} \cite[p. 755f]{Masthoff2015}.