728x90
๋ฐ˜์‘ํ˜•

Machine learning 7

[Machine Learning] Logistic Regression ์˜ˆ์‹œ๋ฅผ ํ†ตํ•ด ๋‹ค์ค‘์„ ํ˜•ํšŒ๊ท€ ์ดํ•ดํ•˜๊ธฐ

Logistic Regression ๋ฒ”์ฃผํ˜• ๋ณ€์ˆ˜ ์˜ˆ์ธก ๋ชจ๋ธ Logistic Regression์„ ์•Œ๊ธฐ์ „์— linear regression์„ ๋จผ์ € ์•Œ์•„์•ผ. Multiple Linear Regression (๋‹ค์ค‘์„ ํ˜•ํšŒ๊ท€) ์ˆ˜์น˜ํ˜• ์„ค๋ช…๋ณ€์ˆ˜ X์™€ ์—ฐ์†ํ˜• ์ˆซ์ž๋กœ ์ด๋ค„์ง„ ์ข…์†๋ณ€์ˆ˜ Y๊ฐ„์˜ ๊ด€๊ณ„๋ฅผ ์„ ํ˜•์œผ๋กœ ๊ฐ€์ •ํ•˜๊ณ  ์ด๋ฅผ ๊ฐ€์žฅ ์ž˜ ํ‘œํ˜„ํ•  ์ˆ˜ ์žˆ๋Š” ํšŒ๊ท€๊ณ„์ˆ˜๋ฅผ ๋ฐ์ดํ„ฐ๋กœ๋ถ€ํ„ฐ ์ถ”์ •ํ•˜๋Š” ๋ชจ๋ธ ์ด๋•Œ ํšŒ๊ท€๊ณ„์ˆ˜๋Š” ๋ชจ๋ธ์˜ ์˜ˆ์ธก๊ฐ’๊ณผ ์‹ค์ œ๊ฐ’์˜ ์ฐจ์ด(์˜ค์ฐจ์ œ๊ณฑํ•ฉ error sum of squared)์„ ์ตœ์†Œ๋กœ ํ•˜๋Š” ๊ฐ’ ์„ค๋ช…๋ณ€์ˆ˜๊ฐ€ p๊ฐœ์ธ ๋‹ค์ค‘์„ ํ˜•ํšŒ๊ท€์˜ ์ผ๋ฐ˜ ์‹ ์˜ˆ์‹œ - 1 ๋‚˜์ด์™€ ํ˜ˆ์•• ๋ฐ์ดํ„ฐ๊ฐ€ ์ฃผ์–ด์กŒ์„ ๋•Œ, ์˜ค์ฐจ์ œ๊ณฑํ•ฉ์„ ์ตœ์†Œ๋กœ ํ•˜๋Š” ํšŒ๊ท€๊ณ„์ˆ˜ ๊ตฌํ•˜๊ธฐ ์„ค๋ช…๋ณ€์ˆ˜ X : ๋‚˜์ด ์ข…์†๋ณ€์ˆ˜ Y : ํ˜ˆ์•• ์•ž์„œ ์ข…์†๋ณ€์ˆ˜ Y๋Š” ‘ํ˜ˆ์••’์œผ๋กœ ์—ฐ์†ํ˜• ์ˆซ์ž์˜€์Œ. ๊ทธ๋ ‡..

[Machine Learning] ์•™์ƒ๋ธ” ๊ธฐ๋ฒ•์ด๋ž€?

Ensemble ๊ธฐ๋ฒ• Ensemble Learning์ด๋ž€ ์—ฌ๋Ÿฌ๊ฐœ์˜ ๋ถ„๋ฅ˜๊ธฐ๋ฅผ ์ƒ์„ฑํ•˜๊ณ  ๊ทธ ์˜ˆ์ธก์„ ๊ฒฐํ•ฉํ•˜์—ฌ ๋ณด๋‹ค ์ •ํ™•ํ•œ ์˜ˆ์ธก์„ ๋‚ด๋Š” ๊ธฐ๋ฒ• ๊ฐ•๋ ฅํ•œ ํ•˜๋‚˜์˜ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜๋Š” ๋Œ€์‹  ๋ณด๋‹ค ์•ฝํ•œ ๋ชจ๋ธ์„ ์—ฌ๋Ÿฌ๊ฐœ ์กฐํ•ฉํ•˜๋Š” ๋ฐฉ์‹ Ensemble Learning ์ข…๋ฅ˜ ์•™์ƒ๋ธ” ํ•™์Šต์€ 3๊ฐ€์ง€ ์œ ํ˜•์œผ๋กœ ๋ถ„๋ฅ˜๋จ Voting Bagging Boosting Voting ์—ฌ๋Ÿฌ๊ฐœ์˜ classifier๊ฐ€ ํˆฌํ‘œ๋ฅผ ํ†ตํ•ด ์ตœ์ข… ์˜ˆ์ธก๊ฒฐ๊ณผ ๊ฒฐ์ • ์„œ๋กœ ๋‹ค๋ฅธ ์•Œ๊ณ ๋ฆฌ์ฆ˜ ์—ฌ๋Ÿฌ๊ฐœ ๊ฒฐํ•ฉํ•˜์—ฌ ์‚ฌ์šฉ Voting ๋ฐฉ์‹ Hard Voting : ๋‹ค์ˆ˜์˜ classifier๊ฐ€ ์˜ˆ์ธกํ•œ ๊ฒฐ๊ณผ๊ฐ’์„ ์ตœ์ข… ๊ฒฐ๊ณผ๋กœ ์„ ์ • (๋‹ค์ˆ˜๊ฒฐ์˜ ๋ฒ•์น™) Soft Voting : ๋ชจ๋“  classifier๊ฐ€ ์˜ˆ์ธกํ•œ label๊ฐ’์˜ ๊ฒฐ์ • ํ™•๋ฅ  ํ‰๊ท ์„ ๊ตฌํ•œ ๋’ค ๊ฐ€์žฅ ํ™•๋ฅ ์ด ๋†’์€ label๊ฐ’์„ ์ตœ์ข…๊ฒฐ๊ณผ๋กœ ์„ ..

[Machine Learning] LightGBM์ด๋ž€? โœ” ์„ค๋ช… ๋ฐ ์žฅ๋‹จ์ 

๐Ÿ“Œ Remind LightGBM์— ๋“ค์–ด๊ฐ€๊ธฐ์ „์— ๋ณต์Šต ๊ฒธ reminding์„ ํ•ด๋ณด์ž. Light GBM์˜ GBM์€ Gradient Boosting Model๋กœ, tree๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ํ•˜๋Š” ํ•™์Šต ์•Œ๊ณ ๋ฆฌ์ฆ˜์ด๋‹ค. ์ด GBM์˜ ํ•™์Šต๋ฐฉ์‹์„ ์‰ฝ๊ฒŒ๋งํ•˜๋ฉด, ํ‹€๋ฆฐ๋ถ€๋ถ„์— ๊ฐ€์ค‘์น˜๋ฅผ ๋”ํ•˜๋ฉด์„œ ์ง„ํ–‰ํ•œ๋‹ค๊ณ  ํ•  ์ˆ˜ ์žˆ๋‹ค. Gradient Boosting์—์„œ Boosting์€ ์—ฌ๋Ÿฌ๊ฐœ์˜ tree๋ฅผ ๋งŒ๋“ค๋˜, ๊ธฐ์กด์— ์žˆ๋Š” ๋ชจ๋ธ(tree)๋ฅผ ์กฐ๊ธˆ์”ฉ ๋ฐœ์ „์‹œ์ผœ์„œ ๋งˆ์ง€๋ง‰์— ์ด๋ฅผ ํ•ฉํ•˜๋Š” ๊ฐœ๋…์œผ๋กœ, Random Forest์˜ Bagging๊ธฐ๋ฒ•๊ณผ ๋‹ค๋ฅธ ๋ฐฉ๋ฒ•์ด๋‹ค. Boostingํ•˜๋Š” ๋ฐฉ์‹์—๋„ ํฌ๊ฒŒ 2๊ฐ€์ง€๊ฐ€ ์žˆ๋‹ค. 1. AdaBoost์™€ ๊ฐ™์ด ์ค‘์š”ํ•œ ๋ฐ์ดํ„ฐ(์ผ๋ฐ˜์ ์œผ๋กœ ๋ชจ๋ธ์ด ํ‹€๋ฆฐ ๋ฐ์ดํ„ฐ)์— ๋Œ€ํ•ด weight๋ฅผ ์ฃผ๋Š” ๋ฐฉ์‹ 2. GBDT์™€ ๊ฐ™์ด loss fun..

[Machine Learning] ๋จธ์‹ ๋Ÿฌ๋‹, ๋ชจ๋ธ์˜ ํŽธํ–ฅ(bias)๊ณผ ๋ถ„์‚ฐ(variance) : trade-off ๊ด€๊ณ„

๋จธ์‹ ๋Ÿฌ๋‹์—์„œ ํŽธํ–ฅ๊ณผ ๋ถ„์‚ฐ์€ ์–ธ์ œ ์“ฐ์ด๋Š” ์šฉ์–ด์ธ๊ฐ€? Supervised Learning(์ง€๋„ํ•™์Šต)์— ๋Œ€ํ•ด์„œ ๊ฐ„๋‹จํžˆ ์„ค๋ช…ํ•ด๋ณด์ž๋ฉด ์‚ฌ๋žŒ์ด ์ •ํ•ด์ค€ ์ •๋‹ต์ด ์žˆ๊ณ , ์šฐ๋ฆฌ์˜ ๋ชจ๋ธ์€ ๊ทธ ์ •๋‹ต์„ ์ž˜ ๋งž์ถ”๋Š” ๋ฐฉํ–ฅ์œผ๋กœ ํ•™์Šต(training)์„ ํ•œ๋‹ค. ์ด๋•Œ, ํ•™์Šต์„ ํ•˜๋ฉด์„œ ๋ชจ๋ธ์ด ๋‚ด๋†“๋Š” ์˜ˆ์ธก๊ฐ’๋“ค์˜ ๊ฒฝํ–ฅ์„ ํ‘œํ˜„ํ•˜๊ธฐ์œ„ํ•ด ํŽธํ–ฅ๊ณผ ๋ถ„์‚ฐ์ด๋ผ๋Š” ์šฉ์–ด๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค. ์‰ฝ๊ฒŒ ๋งํ•˜์ž๋ฉด, ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค A. ์˜ˆ์ธก๊ฐ’๊ณผ ์ •๋‹ต ๊ฐ„์˜ ๊ด€๊ณ„๋ฅผ "ํŽธํ–ฅ"์œผ๋กœ ํ‘œํ˜„ (bias : model์˜ output๊ณผ ์‹ค์ œ๊ฐ’ ์‚ฌ์ด์˜ ์ œ๊ณฑ error, ์ •ํ™•๋„์™€ ๋น„์Šทํ•œ ๊ฐœ๋…) B. ์˜ˆ์ธก๊ฐ’๋ผ๋ฆฌ์˜ ๊ด€๊ณ„๋ฅผ "๋ถ„์‚ฐ"์œผ๋กœ ํ‘œํ˜„ (variance : model์ด ๊ฐ๊ธฐ ๋‹ค๋ฅธ train set์— ๋Œ€ํ•˜์—ฌ ์„ฑ๋Šฅ์˜ ๋ณ€ํ™”์ •๋„๊ฐ€ ๊ธ‰ํ•˜๊ฒŒ ๋ณ€ํ•˜๋Š”์ง€, ์•ˆ์ •์ ์œผ๋กœ ๋ณ€ํ•˜๋Š”์ง€๋ฅผ ๋‚˜ํƒ€๋‚ด๋Š” ์ฒ™๋„) [๋”ฅ๋Ÿฌ๋‹] Bia..

Decision Tree ๊ฐ„.๋‹จ.๋ช….๋ฃŒ

Decision tree : ์˜์‚ฌ๊ฒฐ์ •๋‚˜๋ฌด ๋ถ„๋ฅ˜(classification)๊ณผ ํšŒ๊ท€๋ถ„์„(regression)์— ๋ชจ๋‘ ์‚ฌ์šฉ๋  ์ˆ˜ ์žˆ๊ธฐ ๋–„๋ฌธ์— CART(Classification And Regression Tree)๋ผ๊ณ  ๋ถˆ๋ฆผ node tree์˜ node : ์งˆ๋ฌธ/๋‹ต์„ ๋‹ด๊ณ  ์žˆ์Œ root node : ์ตœ์ƒ์œ„ node ์ตœ์ƒ์œ„ node์˜ ์†์„ฑ feature๊ฐ€ ๊ฐ€์žฅ ์ค‘์š”ํ•œ ํŠน์„ฑ leaf node : ๋งˆ์ง€๋ง‰ node (๋ง๋‹จ๋…ธ๋“œ) ๋งŒ์•ฝ tree์˜ ๋ชจ๋“  leaf node๊ฐ€ pure node๊ฐ€ ๋  ๋•Œ๊นŒ์ง€ ์ง„ํ–‰ํ•˜๋ฉด model์˜ ๋ณต์žก๋„๋Š” ๋งค์šฐ ๋†’์•„์ง€๊ณ  overfitting๋จ overfitting ๋ฐฉ์ง€ tree์˜ ์ƒ์„ฑ์„ ์‚ฌ์ „์— ์ค‘์ง€ : pre-prunning (=๊นŠ์ด์˜ ์ตœ๋Œ€๋ฅผ ์„ค์ •, max_depth) ๋ฐ์ดํ„ฐ๊ฐ€ ์ ์€ node ์‚ญ..

Random Forest ๊ฐ„.๋‹จ.๋ช….๋ฃŒ

Ensemble ์•™์ƒ๋ธ” ์—ฌ๋Ÿฌ ๊ฐœ์˜ ๋จธ์‹ ๋Ÿฌ๋‹ model์„ ์—ฐ๊ฒฐํ•˜์—ฌ ๊ฐ•๋ ฅํ•œ model์„ ๋งŒ๋“œ๋Š” ๊ธฐ๋ฒ• classifier/regression์— ์ „๋ถ€ ํšจ๊ณผ์  random forest์™€ gradient boosting์€ ๋‘˜๋‹ค model์„ ๊ตฌ์„ฑํ•˜๋Š” ๊ธฐ๋ณธ ์š”์†Œ๋กœ decision tree๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค random forest ์กฐ๊ธˆ์”ฉ ๋‹ค ๋‹ค๋ฅธ ์—ฌ๋Ÿฌ decision tree์˜ ๋ฌถ์Œ ๋žœ๋ค ํฌ๋ ˆ์ŠคํŠธ์˜ ๋“ฑ์žฅ ๋ฐฐ๊ฒฝ : ๊ฐ๊ฐ์˜ tree๋Š” ๋น„๊ต์  ์˜ˆ์ธก์„ ์ž˜ ํ•  ์ˆ˜ ์žˆ์ง€๋งŒ, ๋ฐ์ดํ„ฐ์˜ ์ผ๋ถ€์— overfittingํ•˜๋Š” ๊ฒฝํ–ฅ์„ ๊ฐ€์ง ๋”ฐ๋ผ์„œ, ์ž˜ ์ž‘๋™ํ•˜์ง€๋งŒ ์„œ๋กœ ๋‹ค๋ฅธ ๋ฐฉํ–ฅ์œผ๋กœ overfitting๋œ tree๋ฅผ ๋งŽ์ด ๋งŒ๋“ค๊ณ  ๊ทธ ๊ฒฐ๊ณผ๋ฅผ ํ‰๊ท ๋‚ด๋ฉด overfitting์„ ์ค„์ผ ์ˆ˜ ์žˆ๋‹ค ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด tree model์˜ ์˜ˆ์ธก ์„ฑ๋Šฅ์€ ์œ ์ง€ํ•˜๋˜ overf..

๋จธ์‹ ๋Ÿฌ๋‹/AI์—์„œ ์‚ฌ์šฉ๋˜๋Š” "Ground Truth" ๋œป

๋จธ์‹ ๋Ÿฌ๋‹์— ๊ด€๋ จํ•œ ๊ธ€์„ ์ฝ๋‹ค๋ณด๋ฉด "ground-truth"๋ผ๋Š” ์šฉ์–ด๋ฅผ ๋งŽ์ด ์ ‘ํ•˜๊ฒŒ ๋œ๋‹ค. "ground-truth"๋Š” ๊ธฐ์ƒํ•™์—์„œ ์œ ๋ž˜ํ•˜์˜€์œผ๋ฉฐ, ์–ด๋Š ํ•œ ์žฅ์†Œ์—์„œ ์ˆ˜์ง‘๋œ ์ •๋ณด๋ฅผ ์˜๋ฏธํ•˜๋Š” ์šฉ์–ด๋กœ ์‚ฌ์šฉ๋˜์—ˆ๋‹ค๊ณ  ํ•œ๋‹ค. ๊ทธ๋ ‡๋‹ค๋ฉด machine learning/AI์˜ ๋ฌธ๋งฅ์—์„œ ์‚ฌ์šฉ๋˜๋Š” "ground-truth"์˜ ๋œป์€ ๋ฌด์—‡์ธ๊ฐ€. What is ground truth? Ground truth isn't true. It's an ideal expected result. It might involve hand-labeling example datapoints to collect desirable answer data for training your system. For example, a set of images mig..

728x90
๋ฐ˜์‘ํ˜•