Multivariate regression models provide an opportunity to study the effects of multiple independent variables (multiple factors) and their interactions on a single dependent variable. Through multivariate models, it is also possible to build many simplified models at the same time – univariate (single-factor) models. Information about which model we want to build (multivariate or univariate) is visible in the window of the selected analysis. When multiple independent variables are selected simultaneously in the analysis window, it is possible to choose the model.

Preparation of variables for analysis in multivariate models

Variable coding in multivariate models

When to use:

A challenge in preparing data for multivariate analysis is the appropriate coding of nominal and ordinal variables. This is an important element of data preparation, since it has a fundamental impact on the interpretation of the model coefficients. Nominal or ordinal variables divide the analysed objects into two or more categories, with dichotomous variables (with two categories, \(k=2\)) only needing to be coded appropriately, while variables with many categories (\(k>2\)) need to be broken down into dummy variables with two categories and coded.

How to set up the analysis

\(k=2\)

If a variable is dichotomous, the researcher decides how to enter the data representing it, so any numerical codes can be entered, e.g. 0 and 1. In the program, one's own coding can be changed into effect coding by selecting this option in the window of the selected multivariate analysis. This coding replaces the smaller value with -1, and the larger value with 1.

\(k>2\)

If a variable has many categories, in the window of the selected multivariate analysis we select the Dummy variables button and set the reference/base category for those variables that we want to break down into dummy variables. These variables will be dummy-coded, unless the effect coding option is selected in the analysis window – in which case they will be coded as -1, 0, and 1.

Definitions of measures

Dummy coding

is used to answer, with the help of multivariate models, the question: how do the results (\(Y\)) in each analysed category differ from the results of the reference category? This coding consists in assigning a value of 0 or 1 to each category of a given variable. The category coded as 0 is then the reference category.

\(k=2\)

When the coded variable is dichotomous, placing it in a regression model gives us a calculated coefficient (\(b_i\)). This coefficient is the reference of the value of the dependent variable \(Y\) for category 1 to the reference category (adjusted for the other variables in the model).

\(k>2\)

When the analysed variable has more than two categories, \(k\) categories are represented by \(k-1\) dummy variables with dummy coding. When creating dummy-coded variables, a category is selected for which no dummy variable is created. This category is treated in the model as the reference category (since in every dummy-coded variable it corresponds to a value of 0).
[0.2cm] When the variables \(X_1, X_2, ..., X_{k-1}\) obtained this way, with dummy coding, are placed in a regression model, coefficients \(b_1, b_2, ..., b_{k-1}\) will be calculated for them.

\(b_1\)

is the reference of the \(Y\) results (for codes 1 in \(X_1\)) to the reference category (adjusted for the other variables in the model);

\(b_2\)

is the reference of the \(Y\) results (for codes 1 in \(X_2\)) to the reference category (adjusted for the other variables in the model);

...

\(b_{k-1}\)

is the reference of the \(Y\) results (for codes 1 in \(X_{k-1}\)) to the reference category (adjusted for the other variables in the model).

Example
We will code the sex variable, with two categories (male sex will be selected as the reference category), and the education variable, with 4 categories (primary education will be selected as the reference category), using dummy coding.

	Coded
Sex	sex
f	1
f	1
f	1
m	0
m	0
f	1
f	1
m	0
m	0
f	1
m	0
f	1
m	0
f	1
m	0
m	0
...	...

	Coded education
Education	vocational	secondary	tertiary
primary	0	0	0
primary	0	0	0
primary	0	0	0
vocational	1	0	0
vocational	1	0	0
vocational	1	0	0
vocational	1	0	0
secondary	0	1	0
secondary	0	1	0
secondary	0	1	0
secondary	0	1	0
tertiary	0	0	1
tertiary	0	0	1
tertiary	0	0	1
tertiary	0	0	1
tertiary	0	0	1
...	...	...	...

Building on the basis of dummy variables, in a multiple regression model, we might want to check how these variables affect a certain dependent variable, e.g. \(Y\) = the amount of earnings (expressed in thousands of PLN). As a result of such an analysis, we obtain, for each dummy variable, coefficients such as:
– for sex, a statistically significant coefficient \(b_{i}=-0.5\) – meaning that women's average earnings are half a thousand PLN lower than men's, assuming that the other variables in the model remain unchanged;
– for vocational education, a statistically significant coefficient \(b_{i}=0.6\) – meaning that the average earnings of people with vocational education are 0.6 thousand PLN higher than for people with primary education, assuming that the other variables in the model remain unchanged;
– for secondary education, a statistically significant coefficient \(b_{i}=1\) – meaning that the average earnings of people with secondary education are a thousand PLN higher than for people with primary education, assuming that the other variables in the model remain unchanged;
– for tertiary education, a statistically significant coefficient \(b_{i}=1.5\) – meaning that the average earnings of people with tertiary education are 1.5 thousand higher than for people with primary education, assuming that the other variables in the model remain unchanged.

Effect coding

is used to answer, with the help of multivariate models, the question: how do the results (\(Y\)) in each analysed category differ from the (unweighted) sample mean? This coding consists in assigning a value of -1 or 1 to each category of a given variable. The category coded as -1 is then the base category.

\(k=2\)

When the coded variable is dichotomous, placing it in a regression model gives us a calculated coefficient (\(b_i\)). This coefficient is the reference of \(Y\) for category 1 to the unweighted overall mean (adjusted for the other variables in the model).

When the analysed variable has more than two categories, \(k\) categories are represented by \(k-1\) dummy variables with effect coding. When creating variables with effect coding, a category is selected for which no separate variable is created. This category is treated in the model as the base category (since in every variable recorded with effect coding it corresponds to a value of -1).
[0.2cm] When the variables \(X_1, X_2, ..., X_{k-1}\) obtained this way, with effect coding, are placed in a regression model, coefficients \(b_1, b_2, ..., b_{k-1}\) will be calculated for them.

\(b_1\)

is the reference of the \(Y\) results (for codes 1 in \(X_1\)) to the unweighted overall mean (adjusted for the other variables in the model);

\(b_2\)

is the reference of the \(Y\) results (for codes 1 in \(X_2\)) to the unweighted overall mean (adjusted for the other variables in the model);

...

\(b_{k-1}\)

is the reference of the \(Y\) results (for codes 1 in \(X_{k-1}\)) to the unweighted overall mean (adjusted for the other variables in the model).

Example
We will code, using effect coding, the sex variable with two categories (male sex will be selected as the base category), and a variable indicating the region of residence within the country analysed. Five regions are distinguished: north, south, east, west, and central – the central region will be selected as the base category.

	Coded
Sex	sex
f	1
f	1
f	1
m	-1
m	-1
f	1
f	1
m	-1
m	-1
f	1
m	-1
f	1
m	-1
f	1
m	-1
m	-1
...	...

Region	Coded region
of residence	west	east	north	south
central	-1	-1	-1	-1
central	-1	-1	-1	-1
central	-1	-1	-1	-1
west	1	0	0	0
west	1	0	0	0
west	1	0	0	0
west	1	0	0	0
east	0	1	0	0
east	0	1	0	0
east	0	1	0	0
east	0	1	0	0
north	0	0	1	0
north	0	0	1	0
south	0	0	0	1
south	0	0	0	1
south	0	0	0	1
...	...	...	...	...

Building on the basis of dummy variables, in a multiple regression model, we might want to check how these variables affect a certain dependent variable, e.g. \(Y\) = the amount of earnings (expressed in thousands of PLN). As a result of such an analysis, we obtain, for each dummy variable, coefficients such as:
– for sex, a statistically significant coefficient \(b_{i}=-0.5\) – meaning that women's average earnings are half a thousand PLN lower than the average earnings in the country, assuming that the other variables in the model remain unchanged;
– for the western region, a statistically significant coefficient \(b_{i}=0.6\) – meaning that the average earnings of people living in the western region of the country are 0.6 thousand PLN higher than the average earnings in the country, assuming that the other variables in the model remain unchanged;
– for the eastern region, a statistically significant coefficient \(b_{i}=-1\) – meaning that the average earnings of people living in the eastern region of the country are a thousand PLN lower than the average earnings in the country, assuming that the other variables in the model remain unchanged;
– for the northern region, a statistically significant coefficient \(b_{i}=0.4\) – meaning that the average earnings of people living in the north are 0.4 thousand higher than the average earnings in the country, assuming that the other variables in the model remain unchanged;
– for the southern region, a non-statistically-significant coefficient \(b_{i}=0.1\) – meaning that the average earnings of people living in the south do not differ significantly from the average earnings in the country, assuming that the other variables in the model remain unchanged.

Interactions

Interactions are considered in multivariate models, and their presence means that the effect of an independent variable (\(X_1\)) on the dependent variable (\(Y\)) differs depending on the level of another independent variable (\(X_2\)) or a series of further independent variables. To be able to consider interactions in multivariate models, it is necessary to indicate the variables representing the probable interactions, i.e. the products of the relevant variables. For this purpose, we select the Interactions button in the window of the selected multivariate analysis. In the interaction settings window, with the CTRL button held, we select the variables that are to form interactions and move them to the neighbouring list using the arrow. Pressing the OK button gives us the appropriate columns in the data sheet.

In models with interactions, remember to trim them appropriately, so that when removing main effects, the higher-order effects that depend on them are also removed. That is: if a model contains the following variables (main effects): \(X_1\), \(X_2\), \(X_3\), and interactions: \(X_1*X_2\), \(X_1*X_3\), \(X_2*X_3\), \(X_1*X_2*X_3\), then removing variable \(X_1\) from the model also requires removing the interactions in which it appears, i.e. \(X_1*X_2\), \(X_1*X_3\), and \(X_1*X_2*X_3\).