Introduction

This notebook describes and implements a basic approach to solving the Titanic Survival Prediction problem. The prediction is made using a Random Forest Classifier.

1. Exploring training and test sets

First, load required packages.

In?[1]:

importreimportnumpyasnpimportpandasaspdimportmatplotlib.pyplotaspltimportwarningsfromsklearn.ensembleimportRandomForestClassifierwarnings.filterwarnings("ignore")plt.style.use('ggplot')

Read training and test sets. Both datasets will be used in exploring and predicting.

In?[2]:

train=pd.read_csv("../input/train.csv")test=pd.read_csv("../input/test.csv")

In?[3]:

train.sample(frac=1).head(3)

Out[3]:

PassengerIdSurvivedPclassNameSexAgeSibSpParchTicketFareCabinEmbarked

72372402Hodges, Mr. Henry Pricemale50.00025064313.0000NaNS

252613Asplund, Mrs. Carl Oscar (Selma Augusta Emilia...female38.01534707731.3875NaNS

74574601Crosby, Capt. Edward Giffordmale70.011WE/P 573571.0000B22S

In?[4]:

test.sample(frac=1).head(3)

Out[4]:

PassengerIdPclassNameSexAgeSibSpParchTicketFareCabinEmbarked

24711392Drew, Mr. James Vivianmale42.0112822032.500NaNS

29111833Daly, Miss. Margaret Marcella Maggie""female30.0003826506.950NaNQ

58973Svensson, Mr. Johan Cervinmale14.00075389.225NaNS

2. Exploring missing data

Looks like there are missing (NaN) values among both datasets.

In?[5]:

train.info()

RangeIndex: 891 entries, 0 to 890

Data columns (total 12 columns):

PassengerId? ? 891 non-null int64

Survived? ? ? 891 non-null int64

Pclass? ? ? ? 891 non-null int64

Name? ? ? ? ? 891 non-null object

Sex? ? ? ? ? ? 891 non-null object

Age? ? ? ? ? ? 714 non-null float64

SibSp? ? ? ? ? 891 non-null int64

Parch? ? ? ? ? 891 non-null int64

Ticket? ? ? ? 891 non-null object

Fare? ? ? ? ? 891 non-null float64

Cabin? ? ? ? ? 204 non-null object

Embarked? ? ? 889 non-null object

dtypes: float64(2), int64(5), object(5)

memory usage: 83.6+ KB

In?[6]:

test.info()

RangeIndex: 418 entries, 0 to 417

Data columns (total 11 columns):

PassengerId? ? 418 non-null int64

Pclass? ? ? ? 418 non-null int64

Name? ? ? ? ? 418 non-null object

Sex? ? ? ? ? ? 418 non-null object

Age? ? ? ? ? ? 332 non-null float64

SibSp? ? ? ? ? 418 non-null int64

Parch? ? ? ? ? 418 non-null int64

Ticket? ? ? ? 418 non-null object

Fare? ? ? ? ? 417 non-null float64

Cabin? ? ? ? ? 91 non-null object

Embarked? ? ? 418 non-null object

dtypes: float64(2), int64(4), object(5)

memory usage: 36.0+ KB

Non-numeric data

Cabincolumn stores quite a lot of different qualitative values and has a relatively large amount of missing data.

In?[7]:

missing_val_df=pd.DataFrame(index=["Total","Unique Cabin","Missing Cabin"])forname,dfinzip(("Training data","Test data"),(train,test)):total=df.shape[0]unique_cabin=len(df["Cabin"].unique())missing_cabin=df["Cabin"].isnull().sum()missing_val_df[name]=[total,unique_cabin,missing_cabin]missing_val_df

Out[7]:

Training dataTest data

Total891418

Unique Cabin14877

Missing Cabin687327

We shall removeCabincolumns from our dataframes.

Also, we can excludePassengerIdfrom the training set, since IDs are unnecessary for classification.

In?[8]:

train.drop("PassengerId",axis=1,inplace=True)fordfintrain,test:df.drop("Cabin",axis=1,inplace=True)

Fill in missing rows inEmbarkedcolumn withS(Southampton Port), since it's the most frequent.

In?[9]:

non_empty_embarked=train["Embarked"].dropna()unique_values,value_counts=non_empty_embarked.unique(),non_empty_embarked.value_counts()X=range(len(unique_values))colors=["brown","grey","purple"]plt.bar(left=X,height=value_counts,color=colors,tick_label=unique_values)plt.xlabel("Port of Embarkation")plt.ylabel("Amount of embarked")plt.title("Bar plot of embarked in Southampton, Queenstown, Cherbourg")

Out[9]:

Quantitative data

Consider the distributions of passenger ages and fares (excluding NaN values).

In?[10]:

survived=train[train["Survived"]==1]["Age"].dropna()perished=train[train["Survived"]==0]["Age"].dropna()fig,(ax1,ax2)=plt.subplots(nrows=2,ncols=1)fig.set_size_inches(12,6)fig.subplots_adjust(hspace=0.5)ax1.hist(survived,facecolor='green',alpha=0.75)ax1.set(title="Survived",xlabel="Age",ylabel="Amount")ax2.hist(perished,facecolor='brown',alpha=0.75)ax2.set(title="Dead",xlabel="Age",ylabel="Amount")

Out[10]:

[,

,

]

In?[11]:

survived=train[train["Survived"]==1]["Fare"].dropna()perished=train[train["Survived"]==0]["Fare"].dropna()fig,(ax1,ax2)=plt.subplots(nrows=2,ncols=1)fig.set_size_inches(12,8)fig.subplots_adjust(hspace=0.5)ax1.hist(survived,facecolor='darkgreen',alpha=0.75)ax1.set(title="Survived",xlabel="Age",ylabel="Amount")ax2.hist(perished,facecolor='darkred',alpha=0.75)ax2.set(title="Dead",xlabel="Age",ylabel="Amount")

Out[11]:

[,

,

]

We can clean upAgeandFarecolumns filling in all of the missing values withmedianof all values in the training set.

In?[12]:

fordfintrain,test:df["Embarked"].fillna("S",inplace=True)forfeaturein"Age","Fare":df[feature].fillna(train[feature].mean(),inplace=True)

3. Feature engineering

Converting non-numeric columns

All of the non-numeric features exceptEmbarkedaren't particularly informative.

We shall convertEmbarkedandSexcolumns to numeric because we can't feed non-numeric columns into a Machine Learning algorithm.

In?[13]:

fordfintrain,test:forkey,valueinzip(("S","C","Q"),(0,1,2)):df.loc[df["Embarked"]==key,"Embarked"]=valueforkey,valueinzip(("female","male"),(0,1)):df.loc[df["Sex"]==key,"Sex"]=value

Map every unique ticket to numeric ID value.

In?[14]:

fordfintrain,test:ticket_mapping=dict()tickets=list()timer=0for_,sampleindf.iterrows():ifsample["Ticket"]notinticket_mapping:timer+=1ticket_mapping[sample["Ticket"]]=timertickets.append(timer)df["Ticket"]=tickets

Generating new features

SibSpSibSp+ParchParch+11gives the total number of people in a family.

In?[15]:

fordfintrain,test:df["FamilySize"]=df["SibSp"]+df["Parch"]+1

Extract the passengers' titles (Mr., Mrs., Rev., etc.) from their names.

In?[16]:

fordfintrain,test:titles=list()forrowindf["Name"]:surname,title,name=re.split(r"[,.]",row,maxsplit=2)titles.append(title.strip())df["Title"]=titles

In?[17]:

title=train["Title"]unique_values,value_counts=title.unique(),title.value_counts()X=range(len(unique_values))fig,ax=plt.subplots()fig.set_size_inches(18,10)ax.bar(left=X,height=value_counts,width=0.5,tick_label=unique_values)ax.set_xlabel("Title")ax.set_ylabel("Count")ax.set_title("Passenger titles")ax.grid(color='g',linestyle='--',linewidth=0.5)

Looks like some titles are very rare. Let's map them into related titles.

In?[18]:

fordfintrain,test:forkey,valueinzip(("Mr","Mrs","Miss","Master","Dr","Rev"),list(range(6))):df.loc[df["Title"]==key,"Title"]=valuedf.loc[df["Title"]=="Ms","Title"]=1fortitlein"Major","Col","Capt":df.loc[df["Title"]==title,"Title"]=6fortitlein"Mlle","Mme":df.loc[df["Title"]==title,"Title"]=7fortitlein"Don","Sir":df.loc[df["Title"]==title,"Title"]=8fortitlein"Lady","the Countess","Jonkheer":df.loc[df["Title"]==title,"Title"]=9test["Title"][414]=0

Finally, we get

In?[19]:

train.sample(frac=1).head(10)

Out[19]:

SurvivedPclassNameSexAgeSibSpParchTicketFareEmbarkedFamilySizeTitle

28503Stankovic, Mr. Ivan133.000000002558.6625110

77412Hocking, Mrs. Elizabeth (Eliza Needs)054.0000001360923.0000051

51211McGough, Mr. James Robert136.0000000042926.2875010

46803Scanlan, Mr. James129.699118003987.7250210

12903Ekstrom, Mr. Johan145.000000001216.9750010

85813Baclini, Mrs. Solomon (Latifa Qurban)024.0000000365819.2583141

17503Klasen, Mr. Klas Albin118.000000111607.8542030

82813McCormack, Mr. Thomas Joseph129.699118006427.7500210

60503Lindell, Mr. Edvard Bengtsson136.0000001049815.5500020

75803Theobald, Mr. Thomas Leonard134.000000005988.0500010

4. Prediction

Choose the most informative predictors and randomly split the training data.

In?[20]:

fromsklearn.model_selectionimporttrain_test_splitpredictors=["Pclass","Sex","Age","SibSp","Parch","Ticket","Fare","Embarked","FamilySize","Title"]X_train,X_test,y_train,y_test=train_test_split(train[predictors],train["Survived"])

Build a Random Forest model from the training set and evaluate the mean accuracy on the given test set.

In?[21]:

forest=RandomForestClassifier(n_estimators=100,criterion='gini',max_depth=5,min_samples_split=10,min_samples_leaf=5,random_state=0)forest.fit(X_train,y_train)print("Random Forest score:{0:.2}".format(forest.score(X_test,y_test)))

Random Forest score: 0.81

Examine the feature importances.

In?[22]:

plt.bar(range(len(predictors)),forest.feature_importances_)plt.xticks(range(len(predictors)),predictors,rotation='vertical')

Out[22]:

([,

,

,

,

,

,

,

,

,

],

)

Pick the best features and make a submission.

In?[23]:

predictors=["Title","Sex","Fare","Pclass","Age","Ticket"]clf=RandomForestClassifier(n_estimators=100,criterion='gini',max_depth=5,min_samples_split=10,min_samples_leaf=5,random_state=0)clf.fit(train[predictors],train["Survived"])prediction=clf.predict(test[predictors])submission=pd.DataFrame({"PassengerId":test["PassengerId"],"Survived":prediction})submission.to_csv("submission.csv",index=False)