Starting Salary Prediction Model

대학 기관 특성이 초임 연봉에 미치는 영향 분석 | Predicting early-career salary using institutional characteristics and regression-based modeling

비즈니스 맥락 | Business Context

"어떤 대학을 나왔느냐"가 초임 연봉에 얼마나, 어떻게 영향을 미치는가? 기관 유형, 등록금, STEM 비율, 학생 구성 등 정량화 가능한 기관 특성을 바탕으로 졸업생 초임을 예측하는 회귀 모델을 구축하고, 어떤 변수가 연봉 차이를 가장 잘 설명하는지 분석했습니다.

Which institutional characteristics drive early-career salary outcomes and by how much? This project approaches a question relevant to HR analytics and education consulting: can publicly available institutional data predict graduate salary outcomes? Using features such as institution type, tuition cost, STEM concentration, and student composition, I built and compared regression models to identify the structural drivers of starting salary across 444 U.S. colleges.

Project Objective

This project analyzes how institutional characteristics influence early-career salary outcomes across U.S. colleges.

The goal was to:

• Identify key institutional drivers of salary outcomes
• Evaluate linear vs regularized vs tree-based models
• Assess predictive stability using train/test evaluation

주요 결과 | Key Findings

• 등록금(in-state/out-of-state)과 STEM 전공 비율이 초임 연봉의 핵심 예측 변수로 확인됨

  Tuition-related variables and STEM concentration emerged as the strongest predictors of early-career salary

• 사립대학이 공립대학보다 평균 초임이 높은 경향을 정량적으로 확인

  Private institutions show statistically higher average starting salaries than public institutions

• 다중 선형 회귀 모델이 R² ≈ 0.80으로 가장 높은 예측력을 보임 — 릿지 회귀, 의사결정 트리 대비 우수

  Multivariate linear regression achieved the best performance (R² ≈ 0.80), outperforming Ridge regression and Decision Tree

• 모델 검증: Mount Holyoke College 예측값 $55,720 vs 실제값 $52,736 (오차율 약 5.7%)

  Model validation: predicted $55,720 vs actual $52,736 for Mount Holyoke College (~5.7% error)

모델 비교 요약 | Model Comparison

모델 / Model	Test R²	Test MSE	비고 / Notes
Multivariate Linear Regression	~0.80	~13.2M	✅ 최종 선택 / Best overall
Ridge Regression	Similar to OLS	Slightly more stable	Multicollinearity 완화
Decision Tree Regression	~0.59	~22.8M	비선형 관계 캡처 시도

선택 이유: 선형 모델이 가장 높은 예측 정확도와 해석 가능성을 동시에 제공. 규제(Ridge)는 성능 개선 효과 미미, 트리 모델은 과적합 경향 확인.

Highest predictive accuracy, strong interpretability, and stable generalization

Datasets

• 4개 CSV 파일 병합 후 최종 데이터셋: 444개 대학 × 10개 변수

Source: College Tuition, Diversity, and Pay dataset

• This dataset comprises four CSV files: diversity_school, salary_potential, tuition_cost, and tuition_income.
• https://www.kaggle.com/datasets/jessemostipak/college-tuition-diversity-and-pay?select=diversity_school.csv

전처리 | Data Preprocessing

1. 필터 조건: 재학생 2,000명 이상 기관. Select colleges with at least 2000 enrolled students.
2. 비율 변수 생성 (여학생 비율, 유학생 비율). Engineered proportion variables (women, international).
3. 수치형 변수 표준화. Standardized numerical features.
4. 범주형 변수 인코딩. (type: public/private) Encoded categorical variables .
5. Train/test split: 80% / 20%

After cleaning and merging four datasets:

• Final dataset: 444 institutions × 10 variables
• Target variable: early_career_pay

Variable Used:

• name: The name of each college
• total_enrollment: The number of total enrollment in schools
• women_proportion: The proportion of women in schools
• Foreign_proportion: The proportion of international students in schools
• stem_percent: The proportion of students majoring in STEM fields in schools
• net_cost: Average cost of attendance after scholarship/financial aid
• type: Type of schools (private, public)
• in_state_total: Average total cost (room & board + in-state tuition) for in-state residents in USD
• out_of_state_total: Average total cost (room & board + in-state tuition) for out-of-state residents in USD
• early_career_pay: Starting salary in USD

탐색적 데이터 분석 | Exploratory Data Analysis (EDA)

Before model development, a structured exploratory analysis was conducted to understand variable distributions, relationships, and potential multicollinearity.

Insightful Graphs Generated from the Original Datasets

1. Pie chart of the average proportion of each demographic group

   • This pie chart depicts the average proportion of diversity among the institutions in our data.

2. Boxplot comparing the early career pay across different regions in the US

   • Northeast 지역이 가장 높은 평균 초임, South 지역이 가장 낮음
   • This boxplot tells us that among the four regions in the United States, the Northeast has the highest mean early career pay, whereas the South has the lowest average early career pay.

Plots Generated with the Final Dataset

주요 변수 산점도 | Key Variable Relationships

1. Institution type (public vs private):

   

   • 사립대학 > 공립대학 (평균 초임 기준)
   • Private institutions exhibit higher average starting salaries.

2. Relationship between the proportion of women in schools and early career pay

• The proportion of women shows a moderate correlation with salary outcomes, though this likely reflects institutional characteristics rather than causal gender effects.

3. Relationship between the number of total enrollment in schools and early career pay

• Enrollment size shows weak correlation with salary outcomes.

4. Relationship between the tuition-related variables and early career pay

• Tuition-related variables show strong positive correlation with early-career pay.
• Average cost of attendance after scholarship/financial aid has limited linear impact to early-career pay.

5. Relationship between the proportion of international students in schools and early career pay

• International proportion show limited linear impact.

6. Relationship between the proportion of students in STEM fields and early career pay

• STEM concentration demonstrates moderate-to-strong positive association.

Pair plot of variables

Model Specification & Methodology

Feature Selection

Initial candidate predictors included:

1. total_enrollemnt
2. women_proportion
3. foreign_proprotion
4. Stem_percent
5. Net_cost
6. In_state_total
7. Out_of_state_total
8. type

Referring to the covariance matrix, to reduce overfitting in predictive models and increase predictive power, we selected 6 variables with higher correlations to the predictor variables.

• Covariance Matrix with all 8 variables
• total_enrollment and net_cost demonstrate weak correlation with salary outcomes.

Final Feature Set The final predictors used in modeling:

• women_proportion
• foreign_proportion
• stem_percent
• in_state_total
• out_of_state_total
• type

Target variable:

• early_career_pay

모델링 | Modeling Approach

The dataset was split into training (80%) and testing (20%) subsets.

All numerical features were standardized prior to modeling.

Categorical variable type was encoded using indicator variables.

Three regression-based models were implemented and compared.

Model 1: Multivariate Linear Regression (기준 모델 | Baseline Model)

Purpose:

• Establish interpretable benchmark model.
• Evaluate explanatory strength of institutional variables.

Results:

• Test MSE ≈ 13.2M
• Test R² ≈ 0.80

Interpretation: The linear model explains approximately 80% of variance in early-career salary, indicating strong structural relationships between institutional characteristics and salary outcomes.

Model 2: Ridge Regression (L2 정규화, alpha 범위: 10⁻⁵ ~ 10³, 30개 값 탐색)

Purpose:

• Address potential multicollinearity among tuition variables.
• Improve model stability via L2 regularization.

Method:

1. 30 alpha values logarithmically spaced from 10⁻⁵ to 10³.
2. Model performance evaluated across regularization strengths.

Findings:

• Optimal performance observed at very small alpha values.
• Regularization did not significantly improve performance over OLS.

Interpretation: Multicollinearity exists but does not materially degrade predictive performance.

Model 3: Decision Tree Regression (최적 depth 탐색: 1~20, 최종 depth=4)

Purpose:

• Capture potential non-linear relationships.
• Compare interpretability vs performance trade-off.

Model Selection:

• Tree depth evaluated from 1–20.

  

  

Results:

• Test MSE ≈ 22.8M
• Test R² ≈ 0.59

Optimal depth: 4 (based on MSE and R²)

Interpretation: Although the tree model captures non-linear splits, it did not outperform linear approaches. This suggests that early career salary is primarily explained by relatively stable structural relationships rather than complex non-linear interactions.

Final Decision Tree Regression Model

Practical Application

predicted early-career salary for Mount Holyoke College:

• Total annual cost: $86,702

• International students proportion: 23%

• Women proportion: > 90%

• STEM percent: 40%

  

• Predicted Salary: $55,720

• Actual Salary: $52,736

Model Comparison

Model	Test R²	Test MSE	Performance
Linear Regression	~0.80	~13.2M	Best overall
Ridge Regression	Similar to OLS	Slightly more stable	Comparable
Regression Tree	~0.59	~22.8M	Weaker

Final Model Selection: Multiple Linear Regression

• Highest predictive accuracy
• Strong interpretability
• Stable generalization

Results and Evaluation

• Code Repository: https://github.com/yerimoh-23/MachineLearning-StartingSalaryPredictionModel/blob/main/final_code.ipynb

Limitations

• 데이터셋이 복수 연도를 혼합하고 있어 시계열 효과 통제 불가

  Dataset combines multiple years.

• 미국 내 취업 기준 연봉만 반영 (해외 취업 제외)

  Salary outcomes limited to U.S.-based employment.

• 기관 평균값 사용으로 학과·개인 단위 변동성 미반영

  Institutional averages mask within-school variability.

• 누락 변수 가능성: 대학 선별성(selectivity), 전공별 세분화 등

  Potential omitted variable bias (e.g., selectivity, major-level granularity).

Ethical Considerations

• Institutional-level data reduces individual privacy risk.
• Salary prediction models may reinforce socioeconomic stratification.
• Care must be taken not to interpret correlation as causation.