13. Deep Neural Networks for Wage Prediction#

So far we considered many machine learning method, e.g Lasso and Random Forests, to build a predictive model. In this lab, we extend our toolbox by predicting wages by a neural network.

13.1. Data preparation#

Again, we consider data from the U.S. March Supplement of the Current Population Survey (CPS) in 2015.

This notebook contains an example for teaching.

Simple Case Study using Wage Data from 2015 - proceeding

So far we considered many machine learning method, e.g Lasso and Random Forests, to build a predictive model. In this lab, we extend our toolbox by predicting wages by a neural network.

Data preparation

Again, we consider data from the U.S. March Supplement of the Current Population Survey (CPS) in 2015.

# Import relevant packages
import pandas as pd
import numpy as np
import pyreadr
import os
from urllib.request import urlopen
from sklearn import preprocessing
import patsy

from numpy import loadtxt
from keras.models import Sequential
from keras.layers import Dense
import warnings
warnings.filterwarnings('ignore')

link="https://raw.githubusercontent.com/d2cml-ai/14.388_py/main/data/wage2015_subsample_inference.Rdata"
response = urlopen(link)
content = response.read()
fhandle = open( 'wage2015_subsample_inference.Rdata', 'wb')
fhandle.write(content)
fhandle.close()
result = pyreadr.read_r("wage2015_subsample_inference.Rdata")
os.remove("wage2015_subsample_inference.Rdata")

# Extracting the data frame from rdata_read
data = result[ 'data' ]
n = data.shape[0]

# Import relevant packages for splitting data
import random
import math
# Set Seed
# to make the results replicable (generating random numbers)
np.random.seed(0)
random = np.random.randint(0, data.shape[0], size=math.floor(data.shape[0]))
data["random"] = random
random    # the array does not change 

array([2732, 2607, 1653, ..., 4184, 2349, 3462])

data_2 = data.sort_values(by=['random'])
data_2
wage lwage sex shs hsg scl clg ad mw so ... ne exp1 exp2 exp3 exp4 occ occ2 ind ind2 random
rownames
2223 26.442308 3.274965 1.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 ... 1.0 29.0 8.4100 24.389000 70.728100 340 1 8660 20 0
3467 19.230769 2.956512 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 ... 1.0 33.5 11.2225 37.595375 125.944506 9620 22 1870 5 0
13501 48.076923 3.872802 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 ... 0.0 2.0 0.0400 0.008000 0.001600 3060 10 8190 18 0
15588 12.019231 2.486508 0.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 ... 0.0 29.0 8.4100 24.389000 70.728100 6440 19 770 4 2
16049 39.903846 3.686473 1.0 0.0 0.0 0.0 0.0 1.0 0.0 1.0 ... 0.0 12.0 1.4400 1.728000 2.073600 1820 5 7860 17 2
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
27533 21.634615 3.074295 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 ... 0.0 10.0 1.0000 1.000000 1.000000 1860 5 7870 17 5146
7218 13.461538 2.599837 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 ... 0.0 28.0 7.8400 21.952000 61.465600 4220 14 1170 5 5146
7204 27.403846 3.310683 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 ... 0.0 4.0 0.1600 0.064000 0.025600 1760 5 7460 14 5146
1380 16.695804 2.815157 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 ... 1.0 28.0 7.8400 21.952000 61.465600 2000 6 9570 22 5149
10451 45.528846 3.818346 1.0 0.0 0.0 0.0 1.0 0.0 1.0 0.0 ... 0.0 11.0 1.2100 1.331000 1.464100 1107 3 6990 12 5149

5150 rows × 21 columns

# Create training and testing sample 
data_train = data_2[ : math.floor(n*3/4)]    # training sample
data_test =  data_2[ math.floor(n*3/4) : ]   # testing sample
print(data_train.shape)
print(data_test.shape)

(3862, 21)
(1288, 21)

data_train = data_train.iloc[:, 0:16]
data_test = data_test.iloc[:, 0:16] 
data_test
wage lwage sex shs hsg scl clg ad mw so we ne exp1 exp2 exp3 exp4
rownames
9210 29.807692 3.394766 1.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 22.0 4.84 10.648 23.4256
16484 43.269231 3.767442 1.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 17.0 2.89 4.913 8.3521
16448 24.038462 3.179655 1.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 11.0 1.21 1.331 1.4641
27392 10.097115 2.312250 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 30.0 9.00 27.000 81.0000
11596 8.653846 2.158004 1.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 6.0 0.36 0.216 0.1296
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
27533 21.634615 3.074295 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 10.0 1.00 1.000 1.0000
7218 13.461538 2.599837 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 28.0 7.84 21.952 61.4656
7204 27.403846 3.310683 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 4.0 0.16 0.064 0.0256
1380 16.695804 2.815157 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 28.0 7.84 21.952 61.4656
10451 45.528846 3.818346 1.0 0.0 0.0 0.0 1.0 0.0 1.0 0.0 0.0 0.0 11.0 1.21 1.331 1.4641

1288 rows × 16 columns

# normalize the data
from sklearn.preprocessing import MinMaxScaler

scaler =  MinMaxScaler().fit(data_train)
scaler =  MinMaxScaler().fit(data_test)

# scaler = preprocessing.StandardScaler().fit(data_train)
# scaler = preprocessing.StandardScaler().fit(data_test)

data_train_scaled = scaler.transform(data_train)
data_test_scaled = scaler.transform(data_test)

columns = list(data_train)

data_train_scaled = pd.DataFrame(data_train_scaled, columns = columns)
data_test_scaled = pd.DataFrame(data_test_scaled, columns = columns)
data_test_scaled
wage lwage sex shs hsg scl clg ad mw so we ne exp1 exp2 exp3 exp4
0 0.054875 0.447819 1.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.500000 0.250000 0.125000 0.062500
1 0.082510 0.521313 1.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.386364 0.149277 0.057675 0.022284
2 0.043032 0.405398 1.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.250000 0.062500 0.015625 0.003906
3 0.014412 0.234342 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.681818 0.464876 0.316961 0.216110
4 0.011449 0.203924 1.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.136364 0.018595 0.002536 0.000346
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
1283 0.038097 0.384621 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.227273 0.051653 0.011739 0.002668
1284 0.021318 0.291056 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.636364 0.404959 0.257701 0.163992
1285 0.049940 0.431238 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.090909 0.008264 0.000751 0.000068
1286 0.027958 0.333518 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.636364 0.404959 0.257701 0.163992
1287 0.087149 0.531351 1.0 0.0 0.0 0.0 1.0 0.0 1.0 0.0 0.0 0.0 0.250000 0.062500 0.015625 0.003906

1288 rows × 16 columns

Then, we construct the inputs for our network.

formula_basic = "lwage ~ sex + exp1 + shs + hsg+ scl + clg + mw + so + we"
Y_train, model_X_basic_train = patsy.dmatrices(formula_basic, data_train_scaled, return_type='dataframe')
Y_test, model_X_basic_test = patsy.dmatrices(formula_basic, data_test_scaled, return_type='dataframe')

13.2. Neural Networks#

First, we need to determine the structure of our network. We are using the R/python package keras to build a simple sequential neural network with three dense layers.

model_X_basic_train.shape[1]

10

# define the keras model
model = Sequential()
model.add(Dense(20, input_dim = model_X_basic_train.shape[1], activation = 'relu'))
model.add(Dense(10, activation = 'relu'))
# model.add(Dense(5, activation = 'relu'))

model.add(Dense(1))

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# compile the keras model
opt = keras.optimizers.Adam(learning_rate=0.005)
mse = tf.keras.losses.MeanSquaredError()
mae = tf.keras.metrics.MeanAbsoluteError(name="mean_absolute_error", dtype=None)

Let us have a look at the structure of our network in detail.

model.compile(loss=mse, optimizer= opt , metrics=mae)
model.summary(line_length=None, positions=None, print_fn=None)

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense (Dense)               (None, 20)                220       
                                                                 
 dense_1 (Dense)             (None, 10)                210       
                                                                 
 dense_2 (Dense)             (None, 1)                 11        
                                                                 
=================================================================
Total params: 441
Trainable params: 441
Non-trainable params: 0
_________________________________________________________________

It is worth to notice that we have in total \(441\) trainable parameters.

Now, let us train the network. Note that this takes some computation time. Thus, we are using gpu to speed up. The exact speed-up varies based on a number of factors including model architecture, batch-size, input pipeline complexity, etc.

# fit the keras model on the dataset
num_epochs = 1000

Check this link, to understand batch_size argument

# fit the keras model on the dataset
model.fit(model_X_basic_train, Y_train, epochs=150, batch_size=10)

Epoch 1/150
387/387 [==============================] - 2s 2ms/step - loss: 0.0138 - mean_absolute_error: 0.0891
Epoch 2/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0113 - mean_absolute_error: 0.0812
Epoch 3/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0111 - mean_absolute_error: 0.0806
Epoch 4/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0110 - mean_absolute_error: 0.0804
Epoch 5/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0109 - mean_absolute_error: 0.0796
Epoch 6/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0107 - mean_absolute_error: 0.0791
Epoch 7/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0107 - mean_absolute_error: 0.0789
Epoch 8/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0107 - mean_absolute_error: 0.0787
Epoch 9/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0106 - mean_absolute_error: 0.0783
Epoch 10/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0106 - mean_absolute_error: 0.0785
Epoch 11/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0106 - mean_absolute_error: 0.0787
Epoch 12/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0784
Epoch 13/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0106 - mean_absolute_error: 0.0785
Epoch 14/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0106 - mean_absolute_error: 0.0786
Epoch 15/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0781
Epoch 16/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0106 - mean_absolute_error: 0.0788
Epoch 17/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0783
Epoch 18/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0784
Epoch 19/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0782
Epoch 20/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0784
Epoch 21/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0782
Epoch 22/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0106 - mean_absolute_error: 0.0784
Epoch 23/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0782
Epoch 24/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0785
Epoch 25/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0105 - mean_absolute_error: 0.0783
Epoch 26/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0779
Epoch 27/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0104 - mean_absolute_error: 0.0781
Epoch 28/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0781
Epoch 29/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0776
Epoch 30/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0105 - mean_absolute_error: 0.0782
Epoch 31/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0777
Epoch 32/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0776
Epoch 33/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0776
Epoch 34/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0772
Epoch 35/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0779
Epoch 36/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0779
Epoch 37/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0778
Epoch 38/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0104 - mean_absolute_error: 0.0777
Epoch 39/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0772
Epoch 40/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 41/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0772
Epoch 42/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 43/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 44/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0776
Epoch 45/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 46/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 47/150
387/387 [==============================] - 1s 1ms/step - loss: 0.0103 - mean_absolute_error: 0.0773
Epoch 48/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0773
Epoch 49/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0772
Epoch 50/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0777
Epoch 51/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0774
Epoch 52/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 53/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 54/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 55/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 56/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 57/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 58/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0774
Epoch 59/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 60/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0773
Epoch 61/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 62/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0776
Epoch 63/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0773
Epoch 64/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0103 - mean_absolute_error: 0.0777
Epoch 65/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0776
Epoch 66/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 67/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 68/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0774
Epoch 69/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0772
Epoch 70/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0777
Epoch 71/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 72/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0774
Epoch 73/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 74/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 75/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 76/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 77/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 78/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 79/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 80/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 81/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 82/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 83/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 84/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0777
Epoch 85/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 86/150
387/387 [==============================] - 1s 3ms/step - loss: 0.0102 - mean_absolute_error: 0.0774
Epoch 87/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 88/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0773
Epoch 89/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0776
Epoch 90/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 91/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 92/150
387/387 [==============================] - 1s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 93/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0777
Epoch 94/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 95/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0773
Epoch 96/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 97/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 98/150
387/387 [==============================] - 1s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 99/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 100/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 101/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 102/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 103/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0104 - mean_absolute_error: 0.0778
Epoch 104/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 105/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 106/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0774
Epoch 107/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 108/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 109/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0769
Epoch 110/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0774
Epoch 111/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 112/150
387/387 [==============================] - 1s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 113/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 114/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0103 - mean_absolute_error: 0.0773
Epoch 115/150
387/387 [==============================] - 1s 2ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 116/150
387/387 [==============================] - 1s 1ms/step - loss: 0.0103 - mean_absolute_error: 0.0774
Epoch 117/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 118/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0769
Epoch 119/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0768
Epoch 120/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 121/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 122/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 123/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 124/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0769
Epoch 125/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 126/150
387/387 [==============================] - 1s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 127/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0772
Epoch 128/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 129/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0773
Epoch 130/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 131/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 132/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 133/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 134/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 135/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 136/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0103 - mean_absolute_error: 0.0775
Epoch 137/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 138/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0101 - mean_absolute_error: 0.0766
Epoch 139/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0769
Epoch 140/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0101 - mean_absolute_error: 0.0767
Epoch 141/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0101 - mean_absolute_error: 0.0770
Epoch 142/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0767
Epoch 143/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0101 - mean_absolute_error: 0.0767
Epoch 144/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0101 - mean_absolute_error: 0.0770
Epoch 145/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0769
Epoch 146/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0770
Epoch 147/150
387/387 [==============================] - 1s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 148/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 149/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0771
Epoch 150/150
387/387 [==============================] - 0s 1ms/step - loss: 0.0102 - mean_absolute_error: 0.0769

<keras.callbacks.History at 0x27e13c62b50>

model.metrics_names

['loss', 'mean_absolute_error']

model.evaluate(model_X_basic_test, Y_test, verbose = 0)

[0.01021327544003725, 0.0775848850607872]

pred_nn = model.predict(model_X_basic_test)
pred_nn

41/41 [==============================] - 0s 1ms/step

array([[0.31040493],
       [0.42235863],
       [0.39654243],
       ...,
       [0.3823735 ],
       [0.40649807],
       [0.38370064]], dtype=float32)

import statsmodels.api as sm
import statsmodels.formula.api as smf

resid_basic = (Y_test-pred_nn)**2

MSE_nn_basic = sm.OLS( resid_basic , np.ones( resid_basic.shape[0] ) ).fit().summary2().tables[1].iloc[0, 0:2]
MSE_nn_basic

Coef.       0.010213
Std.Err.    0.000552
Name: const, dtype: float64

R2_nn_basic = 1 - ( MSE_nn_basic[0]/Y_test.var() )
print( f"The R^2 using NN is equal to = {R2_nn_basic[0]}" ) # MSE NN (basic model) 

The R^2 using NN is equal to = 0.1511966373372956