2015年12月26日 星期六

Initial result of top 1000 words

Prof. Liu:

I know this may mean nothing to you. But this means a lot to me and to the dyslexic teenager. 
All thanks to your teaching on python and the modern NLTK tools. We are convinced this list and the method would help dyslexics like him a lot. 
I found the psychological effect is so huge on such a kid, he is willing to spend his precious memory on such a certified list generated by our program.
So am I. I am cheered to have such a result and effect.
It brings the suitable threshold to the teenager with his limited ability of acquiring "new words". 

I OCR the special entrance exams papers from 2004 to 2014 and made a corpus specific for him. 
Since some of the text were scanned. But the result of OCR is pretty good. 

The teenager got very limited time and energy to prepare for such a exam. Like, one or two month.
But he's progressing steadily under my supervision. He now got solid syntax structure concept. Good enough to build legitimate English sentences.  
You can still see the glitches in the list. Like some shorten symbols (im =I'm )and (ii) the Roman numbering was also included. 
But it's ok, we would keep improving. 

I am trying to duplicate and understand the Python code written by Sunya last night. (Yeah, it's nothing to her, she finished it so soon!)
Wishing to make another corpus for a junior high school dyslexics, a younger sister of that dyslexic teenager.
But I know it would be not so effective. The corpus and the vocabulary bank for junior high is so small that such an effort would be useless.
Just for fun and see what would be like.

I'd keep reporting my progress if anything relevant to programming and linguistics.

Noted,
Howard

--
吳浩民 Howard Wu
政治大學英語教學所  102551020@nccu.edu.tw
您可以透過 https://db.tt/XtqMCbI  加入dropbox

我是Ptt以下三個板的板主,歡迎跟我討論這些話題
Battery, Browsers, Scenarist
或是有關Linux、自由軟體、電動車、英語教學、語言學、哲學、西洋文學

program generated result of top 1000 most frequent apeared words in 2004-2014 entrance exams

Here they are
We can find some useless words. 
I got to set a threshold to discard them.

The kids are indeed convinced by the result. 



the(6640)be(5719)to(4628)a(4435)of(3033)in(2746)and(2107)have(1757)that(1444)you(1401)it(1301)for(1105)i(925)on(848)b(828)not(782)c(777)can(776)with(730)will(692)do(656)their(652)people(635)or(613)your(603)from(600)if(586)they(562)this(532)he(530)make(481)at(456)d(453)who(449)use(439)more(428)go(425)his(424)one(423)student(420)an(412)pet(407)we(406)get(388)about(383)which(380)take(371)good(356)by(356)most(346)what(344)how(343)she(342)many(329)all(320)when(303)because(301)internet(300)would(299)d)(298)think(297)up(296)but(289)(a(282)her(280)my(275)there(275)so(271)thing(269)read(268)time(265)even(260)very(259)say(252)know(251)child(249)way(245)some(241)give(236)work(229)write(228)no(225)dog(219)passage(219)live(219)should(216)course(216)than(212)like(212)other(212)year(209)study(207)teacher(199)follow(197)water(195)health(195)eat(194)may(189)only(189)email(189)optimistic(184)school(181)want(181)information(180)help(180)our(179)paint(176)problem(176)new(172)him(172)important(171)find(171)mean(170)elephant(169)much(168)english(168)ask(167)accord(166)likely(162)phone(160)over(159)come(157)day(156)show(155)me(154)always(153)own(152)become(151)class(150)them(148)keep(148)now(147)country(145)while(145)le(145)best(144)benefit(144)better(143)robot(143)owner(142)others(141)dont(139)hard(139)out(139)word(138)()(138)meet(138)world(135)body(135)book(133)interest(132)where(130)number(130)look(130)force(129)begin(127)learn(126)place(126)try(126)education(126)human(125)money(125)each(124)computer(119)animal(118)away(117)possible(117)play(116)four(115)after(115)life(115)save(114)right(114)just(113)story(113)those(112)polite(111)program(111)into(111)industrial(110)brain(109)product(108)communicate(108)watch(108)need(107)any(106)also(106)price(105)these(105)put(105)too(104)friend(103)paragraph(102)feel(102)great(102)such(101)two(101)soon(101)stay(101)please(100)optimism(99)pay(98)cause(98)job(98)grade(97)distance(97)shop(97)case(97)often(97)cell(97)sleep(96)though(96)person(96)then(95)lose(95)bad(94)three(93)nt(93)must(92)speak(92)drink(92)smoke(91)example(91)why(91)change(91)mr(90)old(90)aspirin(90)little(90)u(90)disease(89)every(88)long(88)tell(87)access(87)impress(87)however(86)communication(86)report(85)contact(85)mind(84)seem(84)message(84)text(84)generally(83)increase(83)poor(83)family(83)provide(82)lot(82)month(82)since(82)percent(82)ii(81)happen(81)quite(81)question(81)green(80)worker(80)cost(80)part(80)sell(80)material(80))(80)around(79)whom(79)until(78)claim(78)talk(78)company(78)effect(78)last(78)iii(78)understand(78)through(77)could(77)surprise(76)wait(76)happy(75)five(75)never(74)iv(74)natural(74)first(74)relationship(74)run(74)skill(74)true(73)different(73)globe(73)special(73)whether(73)catch(72)tv(72)search(72)food(72)reach(72)something(72)test(71)goal(71)plant(71)small(70)suffer(70)seat(69)see(69)call(68)free(68)prevent(68)healthy(68)movie(68)raise(68)allow(68)off(67)man(66)remote(66)american(66)offer(65)article(65)th(65)send(65)warm(64)ice(64)worry(64)university(64)addition(64)cream(64)drive(63)technology(63)village(63)let(63)baby(63)instant(63)hear(62)diet(62)usually(62)certain(62)attention(62)although(62)wish(62)result(61)mother(61)pain(61)cannot(61)donate(61)ago(61)accident(60)hand(60)father(60)language(60)yourself(60)later(60)between(60)beyond(60)admit(60)waste(60)machine(59)rain(59)im(59)unique(59)consider(59)someone(59)believe(59)include(59)move(59)purpose(58)sure(58)author(58)except(58)idea(58)earth(57)cold(57)cheat(57)enjoy(56)grammar(56)another(56)picture(56)low(56)accompany(56)hurt(55)habit(55)correct(55)refer(55)automate(55)statement(55)taiwan(55)unless(55)turn(54)listen(54)today(54)carry(54)require(54)ted(54)actually(54)able(54)face(54)neither(54)home(54)spread(53)care(53)same(53)imagination(53)healthier(53)teach(53)heart(53)forgery(52)top(52)medical(52)online(52)develop(52)hour(52)reject(52)yesterday(52)high(52)front(52)practice(52)fix(51)form(51)leave(51)prove(51)week(51)either(51)fear(51)group(51)success(51)order(50)mistake(50)down(50)police(50)tap(50)woman(49)wrong(49)stop(49)minute(49)road(49)action(49)dress(49)build(49)sometimes(49)common(49)sit(49)instance(49)walk(49)answer(48)public(48)line(48)few(48)advisor(48)against(47)stress(47)reaction(47)ive(47)locate(47)wang(46)far(46)improve(46)therefore(46)bite(46)parent(46)attitude(46)organization(46)win(46)respect(46)stomach(45)convenient(45)vital(45)main(45)death(45)favorite(45)again(45)extremely(45)spend(45)step(45)naturally(45)room(44)involve(44)classroom(44)quarter(44)chinese(44)rather(44)late(44)might(44)behind(44)famous(44)department(44)young(44)buy(44)jenny(44)overcome(43)present(43)bring(43)equal(43)reasonable(42)shower(42)letter(42)sentence(42)receiver(42)uganda(42)musical(42)sparse(42)milk(42)car(42)pure(42)factory(42)politely(42)stationary(42)politeness(42)lack(42)liter(42)texting(42)paper(42)else(42)ignore(42)download(42)shampoo(42)conclude(42)value(42)suggest(41)positive(41)permission(41)recent(41)link(41)examination(41)customer(41)contest(41)john(40)pineapple(40)release(40)ownership(40)doctor(40)thailand(40)cosmetic(40)advice(40)weight(40)friday(40)nothing(40)during(40)both(40)application(40)exercise(40)opinion(40)didnt(40)allergy(39)safe(39)genuine(39)son(39)depend(39)name(39)college(39)seal(39)mountain(39)higher(39)seller(39)chemical(39)confuse(39)gallery(39)tire(39)ill(38)support(37)along(37)wont(37)component(37)relate(37)succeed(37)decision(37)a(n)(37)knowledge(37)avoid(37)rope(36)upset(36)discussion(36)magazine(36)remember(36)still(36)longer(36)style(36)popular(36)useful(36)account(36)control(36)system(36)(adapted(36)thought(36)risk(35)replace(35)illness(35)assemble(35)determine(35)appointment(35)corpus(35)register(35)really(35)shall(35)fact(35)affect(34)end(34)third(34)l(34)master(34)inspire(34)protect(34)deal(34)twentieth(34)outside(34)surgery(34)region(34)newspaper(34)real(34)start(34)general(34)store(34)big(34)age(34)attend(34)bangkok(33)earn(33)pessimistic(33)microcomputer(33)besides(33)enough(33)mary(33)prepare(33)promote(33)psychology(33)research(33)imagine(33)kind(33)golf(33)government(33)love(32)balance(32)slightly(32)fell(32)entine(32)energy(32)nine(32)short(32)expect(32)foreign(32)conflict(32)cup(32)similar(32)next(32)greater(32)mechanical(32)(abcd)(32)whatever(32)before(31)library(31)serious(31)environment(31)die(31)across(31)touch(31)phrase(31)task(31)per(31)receive(31)light(31)bear(31)tend(31)slowly(31)sister(31)compassion(30)above(30)here(30)fail(30)obese(30)music(30)focus(30)anything(30)nation(30)theory(30)advance(30)daily(30)direction(30)myself(30)check(30)worse(30)everyone(30)wasnt(30)close(30)restaurant(30)jump(30)bird(30)society(30)exam(30)detail(30)full(29)confident(29)thank(29)asthma(29)large(29)strong(29)due(29)once(29)partial(29)native(29)compete(29)wash(28)matter(28)sue(28)singer(28)flow(28)player(28)several(28)(from(28)teeth(28)mainly(28)summary(28)careful(28)kill(28)cambridge(28)connect(28)fat(28)sense(28)return(28)difficulty(28)plan(28)central(28)activity(28)pound(28)century(28)back(28)process(28)quality(28)soap(28)treat(28)intention(28)antacid(27)contribution(27)nerve(27)trip(27)advantage(27)equipment(27)pas(27)alcohol(27)hitchcock(27)maker(27)alive(27)professor(27)marry(27)train(27)complete(27)((27)hydroelectric(27)employee(27)nuclear(27)importance(27)quit(27)immediately(27)solution(26)rank(26)rich(26)tamkang(26)respectably(26)market(26)effort(26)irrational(26)mad(26)indicate(26)stomp(26)rest(26)snarl(26)calligraphy(26)sound(26)contrast(26)compare(26)least(26)dependent(26)demand(26)eager(26)sight(26)behave(26)rabies(26)depression(26)brushwork(26)delay(26)worldwide(26)buyer(26)acknowledge(26)almost(26)thus(26)largest(26)whale(26)scarcely(26)schedule(26)convince(26)mastermind(26)law(25)finally(25)greatest(25)character(25)visit(25)open(25)anyone(25)ridiculous(25)instrument(25)smile(25)easily(25)reduce(25)feed(25)chart(25)fewer(25)request(25)recycle(24)roddick(24)active(24)strike(24)relax(24)warn(24)anita(24)apart(24)advertise(24)temper(24)welcome(24)skin(24)transportation(24)angry(24)fortunately(24)earlier(24)importantly(24)hardly(24)without(24)summer(24)tuition(24)africa(24)unlucky(24)alike(24)recently(24)couple(24)usefuller(24)leg(24)instead(24)choose(24)jesus(24)upon(24)diploma(24)obtain(24)additional(24)successful(23)devote(23)bread(23)clothe(23)spite(23)trust(23)opportunity(23)dinner(23)jane(23)taipei(23)prefer(23)future(23)among(23)dark(23)traffic(23)regard(23)struggle(23)refuse(23)distinguish(23)audience(23)wouldnt(23)engage(23)impression(23)swallow(23)laugh(23)party(23)well(23)happiness(23)demonstrate(23)side(22)exception(22)gift(22)seaside(22)floor(22)role(22)manager(22)charge(22)patient(22)memory(22)cat(22)shake(22)thai(22)normal(22)freshman(22)resource(22)scientist(22)driver(22)table(22)act(22)describe(22)presentation(22)ticket(22)ten(22)further(22)suddenly(22)faculty(22)resort(22)base(22)overwhelm(22)overlook(22)determination(22)graduation(22)forward(22)exploration(22)golfer(22)comfortable(22)afraid(22)organ(22)requirement(22)joy(22)unite(22)difficult(22)journal(22)saver(21)convenience(21)daughter(21)army(21)browse(21)join(21)violin(21)hop(21)preferable(21)classify(21)lay(21)lifeline(21)network(21)thick(21)responsible(21)marvel(21)resemblance(21)altogether(21)subject(21)website(21)madonna(21)id(21)perform(21)beside(21)satellite(21)easy(21)technological(21)linda(21)night(21)wireless(21)emotional(21)entrust(21)corpusbased(21)boy(21)instal(21)topic(21)truck(21)mail(21)instantly(21)introduce(21)half(21)commonplace(21)black(21)dispute(21)dormitory(21)infrastructure(21)countenance(21)speech(21)modem(21)blog(21)luck(21)fertile(21)horse(21)fill(21)renowned(21)horror(21)fall(21)cable(21)arent(21)excuse(21)conspire(21)comfort(21)backyard(21)bypass(21)respire(21)print(21)copper(21)mobile(21)traditional(21)bulky(21)community(21)simply(21)create(21)fire(21)infect(21)admire(21)error(21)cafe(21)income(21)oclock(21)video(21)communicationhungry(21)degree(21)alaska(21)vehicle(21)perspire(21)conversation(21)retail(21)


--
吳浩民 Howard Wu
政治大學英語教學所  102551020@nccu.edu.tw
您可以透過 https://db.tt/XtqMCbI  加入dropbox

我是Ptt以下三個板的板主,歡迎跟我討論這些話題
Battery, Browsers, Scenarist
或是有關Linux、自由軟體、電動車、英語教學、語言學、哲學、西洋文學

下週要問的中醫症狀

雯:手腳冰冷!
我:喉嚨很癢容易咳嗽,夢多,下午會耳朵痛打呵欠,手腕和手指關節痛



--
吳浩民 Howard Wu
政治大學英語教學所  102551020@nccu.edu.tw
您可以透過 https://db.tt/XtqMCbI  加入dropbox

我是Ptt以下三個板的板主,歡迎跟我討論這些話題
Battery, Browsers, Scenarist
或是有關Linux、自由軟體、電動車、英語教學、語言學、哲學、西洋文學