222000420010@unknown@formal@none@1@S@The neuroscience of theory of mind@@@@1@6@@danf@4-5-2012 222000420020@unknown@formal@none@1@S@The study of social cognition ("people thinking about people") and social neuroscience has exploded in the last few years.@@@@1@19@@danf@4-5-2012 222000420030@unknown@formal@none@1@S@Much of energy -- but by no means all of it -- has focused on Theory of Mind.@@@@1@18@@danf@4-5-2012 222000420031@unknown@formal@none@1@S@"Theory of Mind" is something we are all assumed to have -- that is, we all have a theory that other people's actions are best explained by the fact that they have minds which contain wants, beliefs and desires.@@@@1@39@@danf@4-5-2012 222000420040@unknown@formal@none@1@S@(One good reason for calling this a "theory" is that while we have evidence that other people have minds and that this governs their behavior, none of us actually has proof.@@@@1@31@@danf@4-5-2012 222000420050@unknown@formal@none@1@S@And, in fact, some researchers have been claiming that, although we all have minds, those minds do not necessarily govern our behavior.)@@@@1@22@@danf@4-5-2012 222000420060@unknown@formal@none@1@S@Non-human animals and children under the age of 4 do not appear to have theory of mind, except in perhaps a very limited sense.@@@@1@24@@danf@4-5-2012 222000420070@unknown@formal@none@1@S@This leads to the obvious question: what is different about human brains over the age of 4 that allows us to think about other people's thoughts, beliefs and desires?@@@@1@29@@danf@4-5-2012 222000420080@unknown@formal@none@1@S@It might seem like Theory of Mind is such a complex concept that it would be represented diffusely throughout the brain.@@@@1@21@@danf@4-5-2012 222000420090@unknown@formal@none@1@S@However, in the last half-decade or so, neuroimaging studies have locked in on two different areas of the brain.@@@@1@19@@danf@4-5-2012 222000420100@unknown@formal@none@1@S@One, explored by Jason Mitchell of Harvard, among others, is the medial prefrontal cortex (the prefrontal cortex is, essentially, in the front of your brain. "medial" means it is on the interior surface, where the two hemispheres face each other, rather than on the exterior surface, facing your skull).@@@@1@49@@danf@4-5-2012 222000420110@unknown@formal@none@1@S@The other is the temporoparietal junction (where your parietal and temporal lobes meet), described first in neuroimaging by Rebecca Saxe of MIT and colleagues.@@@@1@24@@danf@4-5-2012 222000420120@unknown@formal@none@1@S@Not surprisingly, there is some debate about which of these brain areas is more important (this breaks down in the rather obvious way) and also what the two areas do.@@@@1@30@@danf@4-5-2012 222000420130@unknown@formal@none@1@S@Mitchell and colleagues tend to favor some version of "simulation theory" -- the idea that people (at least in some situations) guess what somebody else might be thinking by implicitly putting themselves in the other person's shoes.@@@@1@37@@danf@4-5-2012 222000420140@unknown@formal@none@1@S@Saxe does not.@@@@1@3@@danf@4-5-2012 222000420150@unknown@formal@none@1@S@Modulo that controversy, theory of mind has been tied to a couple fairly small and distinct brain regions.@@@@1@18@@danf@4-5-2012 222000420160@unknown@formal@none@1@S@These results have been replicated a number of times now and seem to be robust.@@@@1@15@@danf@4-5-2012 222000420170@unknown@formal@none@1@S@This opens up the possibility, among other things, of studying the cross-species variation in theory of mind, as well as the development of theory of mind as children reach their fourth birthdays.@@@@1@32@@danf@4-5-2012 222000660010@unknown@formal@none@1@S@Publication bias@@@@1@2@@danf@4-5-2012 222000660020@unknown@formal@none@1@S@There is an excellent article on publication bias in Slate today.@@@@1@11@@danf@4-5-2012 222000660030@unknown@formal@none@1@S@There is no question that a number of biases affect what gets published and what doesn't.@@@@1@16@@danf@4-5-2012 222000660040@unknown@formal@none@1@S@Some are good (not publishing bad studies), some are bad (not publishing studies that disprove a pet theory) and some are ambiguous (not publishing papers that "aren't interesting").@@@@1@28@@danf@4-5-2012 222000660050@unknown@formal@none@1@S@The big questions are which biases have the biggest impact on what makes its way into print, and how do you take that into account when evaluating the literature.@@@@1@29@@danf@4-5-2012 222000660060@unknown@formal@none@1@S@Read the Slate article here.@@@@1@5@@danf@4-5-2012 222000890010@unknown@formal@none@1@S@Try this at home: Make your own stereogram@@@@1@8@@danf@4-5-2012 222000890020@unknown@formal@none@1@S@Have you ever wanted to make your own 3D movie?@@@@1@10@@danf@4-5-2012 222000890030@unknown@formal@none@1@S@Your own Magic Eye Stereogram?@@@@1@5@@danf@4-5-2012 222000890040@unknown@formal@none@1@S@This post will teach you to create (and see) your own 3D images.@@@@1@13@@danf@4-5-2012 222000890050@unknown@formal@none@1@S@Magic Eye Stereograms are a relatively new technology, but they grew out of the classic stereograms created in 1838 by Charles Wheatstone.@@@@1@22@@danf@4-5-2012 222000890060@unknown@formal@none@1@S@For those of you who don't know what a stereogram is, the word broadly refers to a 3D-like image produced by presenting different images to each eye.@@@@1@27@@danf@4-5-2012 222000890070@unknown@formal@none@1@S@The theory is pretty straight-forward.@@@@1@5@@danf@4-5-2012 222000890080@unknown@formal@none@1@S@Focus on some object in your room (such as your computer).@@@@1@11@@danf@4-5-2012 222000890090@unknown@formal@none@1@S@Now close one eye, then the other.@@@@1@7@@danf@4-5-2012 222000890100@unknown@formal@none@1@S@The objects in your field of vision should shift relative to one another.@@@@1@13@@danf@4-5-2012 222000890110@unknown@formal@none@1@@The closer or father from you they are (relative to the object you are focusing on), the more they should shift.@@@@0@21@@danf@4-5-2012 222000890120@unknown@formal@none@1@S@When you look at a normal photograph (or the text on this screen), this difference is largely lost.@@@@1@18@@danf@4-5-2012 222000890130@unknown@formal@none@1@S@The objects in the picture are in the same position relative to one another regardless of which eye you are looking through.@@@@1@22@@danf@4-5-2012 222000890140@unknown@formal@none@1@S@However, if a clever engineer rigs up a device so as to show different images to each eye in a way that mimics what happens when you look at natural scenes, you will see the illusion of depth.@@@@1@38@@danf@4-5-2012 222000890150@unknown@formal@none@1@S@For instance, she might present the drawing below on the left to your right eye, and the drawing on the right to your left eye: @@@@1@26@@danf@4-5-2012 222000890151@unknown@formal@none@1@S@If the device is set up so that each picture is lined up perfectly with the other (for instance, if each is in the center of the field of vision of the appropriate eye), you would see the colored Xs in the center at different depths relative to one another.@@@@1@50@@danf@4-5-2012 222000890160@unknown@formal@none@1@S@Why?@@@@1@1@@danf@4-5-2012 222000890170@unknown@formal@none@1@S@The green X shifts the most between the two images, so you know it is either the closest or the farthest away.@@@@1@22@@danf@4-5-2012 222000890180@unknown@formal@none@1@S@Importantly, because it's farther to the left in the image shown to the right eye, it must be closer than the blue or red Xs.@@@@1@25@@danf@4-5-2012 222000890190@unknown@formal@none@1@S@You can demonstrate this to yourself using a pencil.@@@@1@9@@danf@4-5-2012 222000890200@unknown@formal@none@1@S@Hold a pencil perfectly vertical a foot or two in front of your face.@@@@1@14@@danf@4-5-2012 222000890210@unknown@formal@none@1@S@It should still look vertical even if you look with only one eye.@@@@1@13@@danf@4-5-2012 222000890220@unknown@formal@none@1@S@Now, tilt the pencil so that the bottom part points towards your chest (at about a 45 degree angle from the floor).@@@@1@22@@danf@4-5-2012 222000890230@unknown@formal@none@1@S@Close your right eye and move the pencil to the right or the left until the pencil appears to be perfectly vertical.@@@@1@22@@danf@4-5-2012 222000890240@unknown@formal@none@1@S@Now look at the pencil with your right eye instead.@@@@1@10@@danf@4-5-2012 222000890250@unknown@formal@none@1@S@It should appear to slope down diagonally to the left.@@@@1@10@@danf@4-5-2012 222000890260@unknown@formal@none@1@S@That is exactly what is happening in the pictures above.@@@@1@10@@danf@4-5-2012 222000890270@unknown@formal@none@1@S@A device that would fuse these two images for you isn't hard to make, but it's even easier to learn how to fuse them simply by crossing your eyes.@@@@1@29@@danf@4-5-2012 222000890280@unknown@formal@none@1@S@There are two ways of crossing your eyes -- making them point inwards towards your nose, and making them point outwards.@@@@1@21@@danf@4-5-2012 222000890290@unknown@formal@none@1@S@One way will make the green X closer; one will make it farther away.@@@@1@14@@danf@4-5-2012 222000890300@unknown@formal@none@1@@I'll describe how to use the first method, because it's the own I typically use.@@@@0@15@@danf@4-5-2012 222000890310@unknown@formal@none@1@S@Look at the two images and cross your eyes towards your nose.@@@@1@12@@danf@4-5-2012 222000890320@unknown@formal@none@1@S@This should cause each of the images to double.@@@@1@9@@danf@4-5-2012 222000890330@unknown@formal@none@1@S@What you want to do is turn those four images into three by causing the middle two to overlap.@@@@1@19@@danf@4-5-2012 222000890340@unknown@formal@none@1@S@This takes some practice.@@@@1@4@@danf@4-5-2012 222000890350@unknown@formal@none@1@S@Try focusing on the Xs that form the rectangular frames of the images.@@@@1@13@@danf@4-5-2012 222000890360@unknown@formal@none@1@S@Make each of those Xs line up exactly with the corresponding X from the frame of the other image.@@@@1@19@@danf@4-5-2012 222000890370@unknown@formal@none@1@S@If you do this, eventually the two images should fuse into a single image, and you will see the colored Xs in depth.@@@@1@23@@danf@4-5-2012 222000890380@unknown@formal@none@1@S@One tip: I find this harder to do on a computer screen than in print, so you might try printing this out.@@@@1@22@@danf@4-5-2012 222000890390@unknown@formal@none@1@S@That is the basic technique.@@@@1@5@@danf@4-5-2012 222000890400@unknown@formal@none@1@S@You should be able to make your own and play around with it to see what you can do.@@@@1@19@@danf@4-5-2012 222000890410@unknown@formal@none@1@S@For instance, this example has a bar pointing up out of the page, but you can also make a bar point into the page.@@@@1@24@@danf@4-5-2012 222000890420@unknown@formal@none@1@S@You also might try creating more complicated objects.@@@@1@8@@danf@4-5-2012 222000890430@unknown@formal@none@1@S@If you want, you can send me any images you make (coglanglab_AT_gmail_DOT_com), and I will post them (you can try including them as comments, but that is tricky).@@@@1@28@@danf@4-5-2012 222000890440@unknown@formal@none@1@S@One final tip -- you'll need to use a font that has uniform spacing.@@@@1@14@@danf@4-5-2012 222000890450@unknown@formal@none@1@S@Courier will work.@@@@1@3@@danf@4-5-2012 222000890460@unknown@formal@none@1@S@Times will not.@@@@1@3@@danf@4-5-2012 222000890470@unknown@formal@none@1@S@Finally, here's another stereogram that uses a completely different principle.@@@@1@10@@danf@4-5-2012 222000890480@unknown@formal@none@1@S@If you can fuse these images, you should see an illusory white box floating in front of a background of Xs.@@@@1@21@@danf@4-5-2012 222000890490@unknown@formal@none@1@S@In a future post, I'll explain how to make these.@@@@1@10@@danf@4-5-2012 222001130010@unknown@formal@none@1@S@Sure, that's plausible@@@@1@3@@danf@4-5-2012 222001130020@unknown@formal@none@1@S@I am happy to say that the results from the recently revived Video Experiment have been excellent, and while we're still collecting some data just in case, the revised paper should be submitted for publication shortly.@@@@1@36@@danf@4-5-2012 222001130030@unknown@formal@none@1@S@That is one month since we got the reviewer's comments back on the original manuscript, which is a faster turn-around than I've ever managed before.@@@@1@25@@danf@4-5-2012 222001130040@unknown@formal@none@1@S@In the meantime, a lab-mate is running a new online survey called, "How Likely? A Plausibility Study."@@@@1@17@@danf@4-5-2012 222001130060@unknown@formal@none@1@S@The idea goes like this.@@@@1@5@@danf@4-5-2012 222001130070@unknown@formal@none@1@S@We use lots of different types of information to understand what people are saying: Word order, general knowledge, intonation, emotion... and plausibility.@@@@1@22@@danf@4-5-2012 222001130080@unknown@formal@none@1@S@If you hear a restaurant employee ask, "Can I bake your order?" you know that the resulting interpretation is implausible.@@@@1@20@@danf@4-5-2012 222001130100@unknown@formal@none@1@S@It would be much more plausible to ask, "Can I take your order?"@@@@1@13@@danf@4-5-2012 222001130110@unknown@formal@none@1@S@That sounds like common sense, but we still don't have a good idea of how and when plausibility is used in comprehension.@@@@1@22@@danf@4-5-2012 222001130120@unknown@formal@none@1@S@To do research in this area, the first thing we need is some sentences that are more or less plausible than others.@@@@1@22@@danf@4-5-2012 222001130130@unknown@formal@none@1@S@The easy way to do it might be to decide for ourselves what we consider to be plausible and implausible sentences.@@@@1@21@@danf@4-5-2012 222001130140@unknown@formal@none@1@S@However, being people who study language all day, we probably aren't very typical.@@@@1@13@@danf@4-5-2012 222001130150@unknown@formal@none@1@S@The point of this study is to get a range of people to say how plausible they think different sentences are.@@@@1@21@@danf@4-5-2012 222001130160@unknown@formal@none@1@S@Then, these sentences and those ratings can be used in further research.@@@@1@12@@danf@4-5-2012 222001130170@unknown@formal@none@1@S@The survey contains 48 sentences and should take about 10 minutes to do.@@@@1@13@@danf@4-5-2012 222001130180@unknown@formal@none@1@S@You can participate in it by clicking here.@@@@1@8@@danf@4-5-2012 222001550010@unknown@formal@none@1@S@Why it doesn't matter if America falls behind in Science@@@@1@10@@danf@4-5-2012 222001550020@unknown@formal@none@1@S@Earlier this year, an article in the New York Times argued that it doesn't matter that the US is losing its edge in science and research.@@@@1@26@@danf@4-5-2012 222001550030@unknown@formal@none@1@S@In fact, the country can save considerable money by letting other countries do the hard work.@@@@1@16@@danf@4-5-2012 222001550040@unknown@formal@none@1@S@The article went on to explain how this can be viewed as outsourcing: let other, cheaper countries do the basic research, and try to turn that research into products in the US.@@@@1@32@@danf@4-5-2012 222001550050@unknown@formal@none@1@S@Really?@@@@1@1@@danf@4-5-2012 222001550051@unknown@formal@none@1@S@The article quoted research published by the National Academy of Sciences, and while I fully recognize that they know more about this topic, have thought more about this topic, and are no doubt considerably smarter, I'm skeptical.@@@@1@37@@danf@4-5-2012 222001550060@unknown@formal@none@1@S@There are two problems I see.@@@@1@6@@danf@4-5-2012 222001550070@unknown@formal@none@1@S@The first is that just because other countries are picking up the slack doesn't mean there isn't slack.@@@@1@18@@danf@4-5-2012 222001550080@unknown@formal@none@1@S@The second is that I'm not convinced that, in the long term, allowing all the best, most cutting-edge research to take place in other countries is really economically sound.@@@@1@29@@danf@4-5-2012 222001550090@unknown@formal@none@1@S@Being a Good Citizen@@@@1@4@@danf@4-5-2012 222001550091@unknown@formal@none@1@S@The article in question seems to imply that there is some amount of research, X, that needs to be done.@@@@1@20@@danf@4-5-2012 222001550100@unknown@formal@none@1@S@If other countries are willing to do it, then no more is needed.@@@@1@13@@danf@4-5-2012 222001550110@unknown@formal@none@1@S@To make this concrete, as long as one new disease is cured, say, every five years, there's simply no reason to invest any additional energy into curing diseases.@@@@1@28@@danf@4-5-2012 222001550120@unknown@formal@none@1@S@That's enough.@@@@1@2@@danf@4-5-2012 222001550130@unknown@formal@none@1@S@And for people who have some other disease that hasn't been cured, they can wait their turn.@@@@1@17@@danf@4-5-2012 222001550140@unknown@formal@none@1@S@The concept is most clear when it comes to disease, but I think the same argument applies everywhere else.@@@@1@19@@danf@4-5-2012 222001550150@unknown@formal@none@1@S@Basic science is what gives us new technology, and technology has been humanity's method of improving our quality of life since at least a few million years ago.@@@@1@28@@danf@4-5-2012 222001550160@unknown@formal@none@1@S@Perhaps some people think quality of life is improving fast enough -- or too fast, thank you -- but I, at least, would like my Internet connection to be a bit faster now rather than later.@@@@1@36@@danf@4-5-2012 222001550170@unknown@formal@none@1@S@The fact that China, Taiwan, Singapore & co. are stepping up to the plate is not a reason for us to go on vacation.@@@@1@24@@danf@4-5-2012 222001550180@unknown@formal@none@1@S@Can We Really be Competitive as a Backwater?@@@@1@8@@danf@4-5-2012 222001550190@unknown@formal@none@1@S@The article casts "outsourcing" science as good business by noting that America is still the best at turning science into products.@@@@1@21@@danf@4-5-2012 222001550200@unknown@formal@none@1@S@So let other countries do the expensive investment into research -- we'll just do the lucrative part that comes later.@@@@1@20@@danf@4-5-2012 222001550210@unknown@formal@none@1@S@Do they think other countries won't catch on?@@@@1@8@@danf@4-5-2012 222001550220@unknown@formal@none@1@S@I have to imagine that Singapore and similar countries are investing in research because they want to make money.@@@@1@19@@danf@4-5-2012 222001550230@unknown@formal@none@1@S@Which means they will want their share of the lucrative research-to-product business.@@@@1@12@@danf@4-5-2012 222001550240@unknown@formal@none@1@S@So America's business plan, then, would have to be to try to keep our advantage on that front while losing our advantage on basic research.@@@@1@25@@danf@4-5-2012 222001550250@unknown@formal@none@1@S@This may well be possible.@@@@1@5@@danf@4-5-2012 222001550260@unknown@formal@none@1@S@But it has some challenges.@@@@1@5@@danf@4-5-2012 222001550270@unknown@formal@none@1@S@It's no accident that the neighborhood around MIT is packed with tech start-ups.@@@@1@13@@danf@4-5-2012 222001550280@unknown@formal@none@1@S@I'm not a sociologist, but I can speculate on why that is.@@@@1@12@@danf@4-5-2012 222001550290@unknown@formal@none@1@S@First, many of those tech start-ups are founded by MIT graduates.@@@@1@11@@danf@4-5-2012 222001550300@unknown@formal@none@1@S@They aren't necessarily Boston natives, but having been drawn to one of the world's great research universities, they end up settling there.@@@@1@22@@danf@4-5-2012 222001550310@unknown@formal@none@1@S@Second, Flat World or not, there are advantages to being close to the action.@@@@1@14@@danf@4-5-2012 222001550320@unknown@formal@none@1@S@Many non-scientists don't realize that by the time "cutting-edge" research is published, it is often a year or even several years old.@@@@1@22@@danf@4-5-2012 222001550330@unknown@formal@none@1@S@The way to stay truly current is to chat with the researchers over coffee about what they are doing right now, not about what they are writing right now.@@@@1@29@@danf@4-5-2012 222001550340@unknown@formal@none@1@S@Third, science benefits from community.@@@@1@5@@danf@4-5-2012 222001550350@unknown@formal@none@1@S@Harvard's biggest advantage, as far as I can tell, is the existence of MIT two miles down the road, and visa versa.@@@@1@22@@danf@4-5-2012 222001550360@unknown@formal@none@1@S@Waxing poetic about the free exchange of ideas may sound a bit abstract, but it has a real impact.@@@@1@19@@danf@4-5-2012 222001550370@unknown@formal@none@1@S@I have multiple opportunities each week to discuss my current projects with some of the best minds in the field, and I do better work for it.@@@@1@27@@danf@4-5-2012 222001550380@unknown@formal@none@1@S@In short, I think any country that maintains the world's premier scientific community is going to have impressive structural advantages when it comes to converting ideas into money.@@@@1@28@@danf@4-5-2012 222001550390@unknown@formal@none@1@S@That Said...@@@@1@2@@danf@4-5-2012 222001550391@unknown@formal@none@1@S@That said, I think there are two really useful ideas that come out of that article.@@@@1@16@@danf@4-5-2012 222001550400@unknown@formal@none@1@S@The first is the challenge against the orthodoxy that strong science = strong economy.@@@@1@14@@danf@4-5-2012 222001550410@unknown@formal@none@1@S@Without challenges like these, we can't home in on what exactly is important about funding basic research (not saying I've been successful here, but it is a start, at least).@@@@1@30@@danf@4-5-2012 222001550420@unknown@formal@none@1@S@The second is that even if the US maintains its lead in science, that lead is going to shrink no matter what we do, so it's important to think about how to capitalize on discoveries coming in from overseas.@@@@1@39@@danf@4-5-2012 222001550430@unknown@formal@none@1@S@Political Note@@@@1@2@@danf@4-5-2012 222001550431@unknown@formal@none@1@S@Those who are concerned about basic research in the US should note that while John McCain does not list science funding as a priority on his website -- unless you count non-specific support of NASA -- and did not mention it in his convention speech, Barack Obama did both (he supports doubling basic science funding).@@@@1@55@@danf@4-5-2012 222001550432@unknown@formal@none@1@S@Folks in Eastern Washington may be interested to know that a clinical psychologist is running for Congress against an incumbent.@@@@1@20@@danf@4-5-2012 222001550440@unknown@formal@none@1@S@Though Mark Mays has been professionally more involved in treatment than in research, research is among his top priorities.@@@@1@19@@danf@4-5-2012 222001740010@unknown@formal@none@1@S@Science's Call to Arms@@@@1@4@@danf@4-5-2012 222001740020@unknown@formal@none@1@S@In case anyone was wondering, I am far from alone in my call for a new science policy in the coming administration.@@@@1@22@@danf@4-5-2012 222001740030@unknown@formal@none@1@S@It is the topic of the editorial in the latest issue of Science Magazine America's premier scientific journal:@@@@1@18@@danf@4-5-2012 222001740040@unknown@formal@none@1@S@For the past 7 years, the United States has had a presidential administration where science has had little place at the table.@@@@1@22@@danf@4-5-2012 222001740050@unknown@formal@none@1@S@We have had a president opposed to embryonic stem cell research and in favor of teaching intelligent design.@@@@1@18@@danf@4-5-2012 222001740060@unknown@formal@none@1@S@We have had an administration that at times has suppressed, rewritten, ignored, or abused scientific research.@@@@1@16@@danf@4-5-2012 222001740070@unknown@formal@none@1@S@At a time when scientific opportunity has never been greater, we have had five straight years of inadequate increases for U.S. research agencies, which for some like the National Institutes of Health (NIH) means decreases after inflation.@@@@1@37@@danf@4-5-2012 222001740080@unknown@formal@none@1@S@All of this has been devastating for the scientific community; has undermined the future of our economy, which depends on innovation; and has slowed progress toward better health and greater longevity for people around the world.@@@@1@36@@danf@4-5-2012 222001740090@unknown@formal@none@1@S@Dr. Porter, the editorialist, goes on to ask So if you are a U.S. scientist, what should you do now?@@@@1@20@@danf@4-5-2012 222001740100@unknown@formal@none@1@S@He offers a number of ideas, most of which are probably not practical for a graduate student like myself ("volunteer to advise ... candidates on science matters and issues.").@@@@1@29@@danf@4-5-2012 222001740110@unknown@formal@none@1@S@The one that is most practical and which anybody can do is to promote ScienceDebate2008.com.@@@@1@15@@danf@4-5-2012 222001740120@unknown@formal@none@1@@He acknowledges that the program's goal -- a presidential debate dedicated to science -- will not be accomplished in 2008, bu the hope is to signal to the media and to politicians that people care about science and science policy.@@@@0@40@@danf@4-5-2012 222001740130@unknown@formal@none@1@S@And who knows?@@@@1@3@@danf@4-5-2012 222001740140@unknown@formal@none@1@@Maybe there will be a science debate is 2012?@@@@0@9@@danf@4-5-2012 222002090010@unknown@formal@none@1@S@Androids Run Amok at the New York Times?@@@@1@8@@danf@4-5-2012 222002090020@unknown@formal@none@1@S@I have been reading Steve Pinker's excellent essay in the New York Times about the advent of personal genetics.@@@@1@19@@danf@4-5-2012 222002090030@unknown@formal@none@1@S@Reading it, though, I noticed something odd.@@@@1@7@@danf@4-5-2012 222002090040@unknown@formal@none@1@S@The Times includes hyperlinks in most of its articles, usually linking to searches for key terms within its own archive.@@@@1@20@@danf@4-5-2012 222002090050@unknown@formal@none@1@S@I used to think this linking was done by hand, as I do in my own posts.@@@@1@17@@danf@4-5-2012 222002090060@unknown@formal@none@1@S@Lately, I think it's done by an android (and not a very smart one).@@@@1@14@@danf@4-5-2012 222002090070@unknown@formal@none@1@S@Often the links are helpful in the obvious way.@@@@1@9@@danf@4-5-2012 222002090080@unknown@formal@none@1@S@Pinker mentions Kareem Abdul-Jabbar, and the Times helpfully links to a list of recent articles that mention him.@@@@1@18@@danf@4-5-2012 222002090090@unknown@formal@none@1@S@Presumably this is for the people who don't know who he is (though a link to the Abdul-Jabbar Wikipedia entry might be more useful).@@@@1@24@@danf@4-5-2012 222002090100@unknown@formal@none@1@S@Some links are less obvious.@@@@1@5@@danf@4-5-2012 222002090110@unknown@formal@none@1@S@In a sentence that begins "Though health and nutrition can affect stature..." the Time sticks in a hyperlink for articles related to nutrition.@@@@1@23@@danf@4-5-2012 222002090120@unknown@formal@none@1@S@I guess that's in case the word stirs me into wondering what else the Times has written about nutrition.@@@@1@19@@danf@4-5-2012 222002090130@unknown@formal@none@1@S@That can't explain the following sentence though:@@@@1@7@@danf@4-5-2012 222002090140@unknown@formal@none@1@S@Another kind of headache for geneticists comes from gene variants that do have large effects but that are unique to you or to some tiny fraction of humanity.@@@@1@28@@danf@4-5-2012 222002090150@unknown@formal@none@1@S@There is just no way any human thought that readers would want a list of articles from the medical section about headaches.@@@@1@22@@danf@4-5-2012 222002090160@unknown@formal@none@1@S@This suggests that the Times simply has a list of keywords that are automatically tagged in every article...or perhaps it is slightly more sophisticated and the keywords vary based on the section of the paper.@@@@1@35@@danf@4-5-2012 222002090170@unknown@formal@none@1@S@I'm not sure how useful this is even in the best of circumstances.@@@@1@13@@danf@4-5-2012 222002090180@unknown@formal@none@1@S@Has anyone ever actually clicked on one of these links and read any of the articles listed?@@@@1@17@@danf@4-5-2012 222002090190@unknown@formal@none@1@S@If so, comment away!@@@@1@4@@danf@4-5-2012 222002090200@unknown@formal@none@1@S@(picture from Weeklyreader.com)@@@@1@3@@danf@4-5-2012 222002830010@unknown@formal@none@1@S@Games with Words: New Web lab launched@@@@1@7@@danf@4-5-2012 222002830020@unknown@formal@none@1@S@The new Lab is launched (finally).@@@@1@6@@danf@4-5-2012 222002830030@unknown@formal@none@1@S@I was a long ways from the first to start running experiments on the Web.@@@@1@15@@danf@4-5-2012 222002830040@unknown@formal@none@1@S@Nonetheless, when I got started in late 2006, the Web had mostly been used for surveys, and there were only a few examples of really successful Web laboratories (like the Moral Sense Test, FaceResearch and Project Implicit).@@@@1@37@@danf@4-5-2012 222002830050@unknown@formal@none@1@S@There were many examples of failed attempts.@@@@1@7@@danf@4-5-2012 222002830060@unknown@formal@none@1@S@So I wasn't really sure what a Web laboratory should look like, how it could best be utilized, or what would make it attractive and useful for participants.@@@@1@28@@danf@4-5-2012 222002830070@unknown@formal@none@1@S@I put together a website known as Visual Cognition Online for the lab I was working at.@@@@1@17@@danf@4-5-2012 222002830080@unknown@formal@none@1@S@I was intrigued by the possibility of running one-trial experiments.@@@@1@10@@danf@4-5-2012 222002830090@unknown@formal@none@1@S@Testing people involves a lot of noise, so we usually try to get many measurements (sometimes hundreds) from each participant, in order to get a good estimate of what we're trying to measure.@@@@1@33@@danf@4-5-2012 222002830100@unknown@formal@none@1@S@Sometimes this isn't practical.@@@@1@4@@danf@4-5-2012 222002830110@unknown@formal@none@1@S@The best analogy that comes to mind is football.@@@@1@9@@danf@4-5-2012 222002830120@unknown@formal@none@1@S@A lot of luck and random variation goes into each game, so ideally, we'd like each team to play each other several times (like happens in baseball).@@@@1@27@@danf@4-5-2012 222002830130@unknown@formal@none@1@S@However, the physics of football makes this impractical (it'd kill the players).@@@@1@12@@danf@4-5-2012 222002830140@unknown@formal@none@1@S@Running a study on the Web makes it possible to test more participants, which means we don't need as many trials from each.@@@@1@23@@danf@4-5-2012 222002830150@unknown@formal@none@1@S@A few studies worked well enough, and I got other good data along the way (like this project), so when the lab moved to MN and I moved to graduate school, I started the Cognition and Language Lab along the same model.@@@@1@42@@danf@4-5-2012 222002830160@unknown@formal@none@1@S@Web Research blooms@@@@1@3@@danf@4-5-2012 222002830170@unknown@formal@none@1@S@In the last two years, Web research has really taken off, and we've all gotten a better sense of what it was useful for.@@@@1@24@@danf@4-5-2012 222002830180@unknown@formal@none@1@S@The projects that make me most excited are those run by the likes of TestMyBrain.org, Games with a Purpose, and Phrase Detectives.@@@@1@22@@danf@4-5-2012 222002830190@unknown@formal@none@1@S@These sites harness the massive size of the Internet to do work that wasn't just impossible before -- it was frankly inconceivable.@@@@1@22@@danf@4-5-2012 222002830200@unknown@formal@none@1@S@As I understand it, the folks behind Games with a Purpose are mainly interested in machine learning.@@@@1@17@@danf@4-5-2012 222002830210@unknown@formal@none@1@S@They train computer programs to do things, like tag photographs according to content.@@@@1@13@@danf@4-5-2012 222002830220@unknown@formal@none@1@S@To train their computer programs, they need a whole bunch of photographs tagged for content; you can't test a computer -- or a person -- if you don't know what the correct answer is.@@@@1@34@@danf@4-5-2012 222002830230@unknown@formal@none@1@S@Their games are focused around doing things like tagging photographs.@@@@1@10@@danf@4-5-2012 222002830240@unknown@formal@none@1@S@Phrase Detectives does something similar, but with language.@@@@1@8@@danf@4-5-2012 222002830250@unknown@formal@none@1@S@The most exciting results from TestMyBrain.org (full disclosure: the owner is a friend of mine, a classmate at Harvard, and also a collaborator) have focused on the development and aging of various skills.@@@@1@33@@danf@4-5-2012 222002830260@unknown@formal@none@1@S@Normally, when we look at development, we test a few different age groups.@@@@1@13@@danf@4-5-2012 222002830270@unknown@formal@none@1@S@An extraordinarily ambitious project would test some 5 year olds, some 20 year olds, some 50 year olds, and some 80 year olds.@@@@1@23@@danf@4-5-2012 222002830280@unknown@formal@none@1@S@By testing on the Web, they have been able to look at development and aging from the early teenage years through retirement age (I'll blog about some of my own similar work in the near future).@@@@1@36@@danf@4-5-2012 222002830290@unknown@formal@none@1@S@Enter: GamesWithWords.org@@@@1@2@@danf@4-5-2012 222002830291@unknown@formal@none@1@S@This Fall, I started renovating coglanglab.org in order to incorporate some of the things I liked about those other sites.@@@@1@20@@danf@4-5-2012 222002830300@unknown@formal@none@1@S@The project quickly grew, and in the end I decided that the old name (Cognition and Language Lab) just didn't fit anymore.@@@@1@22@@danf@4-5-2012 222002830310@unknown@formal@none@1@S@GamesWithWords.org was born.@@@@1@3@@danf@4-5-2012 222002830320@unknown@formal@none@1@S@I've incorporated many aspects of the other sites that I like.@@@@1@11@@danf@4-5-2012 222002830330@unknown@formal@none@1@S@One is simply to make the site more engaging (reflected, I hope, in the new name).@@@@1@16@@danf@4-5-2012 222002830340@unknown@formal@none@1@S@It's always been my goal to make the Lab interesting and fun for participants (the primary goal of this blog is to explain the research and disseminate results), and I've tried to adopt the best ideas I've seen elsewhere.@@@@1@39@@danf@4-5-2012 222002830350@unknown@formal@none@1@S@Ultimately, of course, the purpose of any experiment is not just to produce data, but to produce good data that tests hypotheses and furthers theory.@@@@1@25@@danf@4-5-2012 222002830360@unknown@formal@none@1@S@This sometimes limits what I can do with experiments (for instance, while I'd love to give individualized feedback to each participant for every experiment, sometimes the design just doesn't lend itself to feedback.@@@@1@33@@danf@4-5-2012 222002830370@unknown@formal@none@1@@Of the two experiments that are currently like, one offers feedback, one doesn't.@@@@0@13@@danf@4-5-2012 222002830380@unknown@formal@none@1@S@I'll be writing more about the new experiments over the upcoming days.@@@@1@12@@danf@4-5-2012 222003720010@unknown@formal@none@1@S@Sounds of Silence@@@@1@3@@danf@4-5-2012 222003720020@unknown@formal@none@1@S@My lament that, with regards to discussion of education reform, a trace of small liberal arts colleges has disappeared into the ether appears to have, itself, disappeared into the ether.@@@@1@30@@danf@4-5-2012 222003720030@unknown@formal@none@1@S@Seriously, readers, I expected some response to that one.@@@@1@9@@danf@4-5-2012 222003720040@unknown@formal@none@1@S@There are parts of my post even I disagree with.@@@@1@10@@danf@4-5-2012 222003810010@unknown@formal@none@1@S@Making data public@@@@1@3@@danf@4-5-2012 222003810020@unknown@formal@none@1@S@Lately, there have been a lot of voices (e.g., this one) calling for scientists to make raw data immediately available to the general public.@@@@1@24@@danf@4-5-2012 222003810030@unknown@formal@none@1@@In the interest of answer than call, here's some of my raw data:@@@@0@13@@danf@4-5-2012 222003810031@unknown@formal@none@1@S@Do you feel enlightened?@@@@1@4@@danf@4-5-2012 222003810040@unknown@formal@none@1@S@Probably not.@@@@1@2@@danf@4-5-2012 222003810050@unknown@formal@none@1@S@Raw data isn't all that useful if you don't know how it was collected, what the different numbers refer to, etc.@@@@1@21@@danf@4-5-2012 222003810060@unknown@formal@none@1@S@Even if I told you this is data from this experiment, that probably wouldn't help much.@@@@1@16@@danf@4-5-2012 222003810070@unknown@formal@none@1@S@Even showing you the header rows for these data will help only so much: Some things are straightforward.@@@@1@18@@danf@4-5-2012 222003810080@unknown@formal@none@1@S@Some are not.@@@@1@3@@danf@4-5-2012 222003810090@unknown@formal@none@1@S@It's important to know that I record data with a separate row for every trial, so each participant has multiple trials.@@@@1@21@@danf@4-5-2012 222003810100@unknown@formal@none@1@S@Also, I record all data, even data from participants who did not complete the experiment.@@@@1@15@@danf@4-5-2012 222003810110@unknown@formal@none@1@S@If you're unaware of that, your data analyses would come out very wrong.@@@@1@13@@danf@4-5-2012 222003810120@unknown@formal@none@1@S@Also I have some codes I use to mark that the participant is an experimenter checking to make sure everything is running correctly.@@@@1@23@@danf@4-5-2012 222003810130@unknown@formal@none@1@S@You'd need to know those.@@@@1@5@@danf@4-5-2012 222003810140@unknown@formal@none@1@S@It's key to know how responses are coded (it's not simply "right" or "wrong" -- and in fact the column called totalCorrect does not record whether the participant got anything correct).@@@@1@31@@danf@4-5-2012 222003810150@unknown@formal@none@1@S@The truth is, even though I designed this study myself and wrote the program that outputs the data, every time I go back to data from a study I haven't worked with in a while, it takes me a few hours to orient myself -- and I'm actually relatively good about documenting my data.@@@@1@54@@danf@4-5-2012 222003810160@unknown@formal@none@1@S@So if a law were passed -- as some have advocated for -- requiring that data be made public, one of two things will happen: either people will post uninterpretable data like my mini-spreadsheet above, or they'll spend huge amounts of time preparing their data for others' consumption.@@@@1@48@@danf@4-5-2012 222003810170@unknown@formal@none@1@S@The former will help no one.@@@@1@6@@danf@4-5-2012 222003810180@unknown@formal@none@1@S@And the latter is expensive, and someone has to pay for that.@@@@1@12@@danf@4-5-2012 222003810190@unknown@formal@none@1@S@And this all has to be balanced against the fact that there are very few data sets anyone would want to reanalyze.@@@@1@22@@danf@4-5-2012 222003810200@unknown@formal@none@1@S@There are important datasets that should be made available.@@@@1@9@@danf@4-5-2012 222003810210@unknown@formal@none@1@S@And in fact there are already mechanisms for doing this (in my field, CHILDES is a good example).@@@@1@18@@danf@4-5-2012 222003810220@unknown@formal@none@1@S@This kind of sharing should be encouraged, but mandated sharing is likely to cause more problems than it solves.@@@@1@19@@danf@4-5-2012 222004210010@unknown@formal@none@1@S@Apply to Graduate School?@@@@1@4@@danf@4-5-2012 222004210020@unknown@formal@none@1@S@Each year around this time, I try to post more information that would be of use to prospective graduate students, just in case any such are reading this blog (BTW Are there any undergraduates reading this blog?@@@@1@37@@danf@4-5-2012 222004210030@unknown@formal@none@1@S@Post in the comments!).@@@@1@4@@danf@4-5-2012 222004210040@unknown@formal@none@1@S@This year, I've been swamped.@@@@1@5@@danf@4-5-2012 222004210050@unknown@formal@none@1@S@I've been focusing on getting a few papers published, and most of my time for blogging has gone to the Scientific-American-Mind-article-that-will-not-die, which, should I ever finish it, will probably come out early next year.@@@@1@34@@danf@4-5-2012 222004210060@unknown@formal@none@1@S@Luckily, Female Science Professor has written a comprehensive essay on The Chronicle of Higher Education about one of the most confusing parts of the application process: the pre-application email to a potential advisor.@@@@1@33@@danf@4-5-2012 222004210070@unknown@formal@none@1@S@Everyone tells applicants to send such emails, but nobody gives much information about what should be in them.@@@@1@18@@danf@4-5-2012 222004210080@unknown@formal@none@1@S@Find the essay here.@@@@1@4@@danf@4-5-2012 222004210090@unknown@formal@none@1@S@I would add one comment to what she wrote.@@@@1@9@@danf@4-5-2012 222004210100@unknown@formal@none@1@S@She points out that you should check the website to see what kind of research the professor does rather than just asking, "Can you tell me more about your research," which comes across as lazy.@@@@1@35@@danf@4-5-2012 222004210110@unknown@formal@none@1@S@She also suggests that you should put in your email whether you are interested in a terminal master's.@@@@1@18@@danf@4-5-2012 222004210120@unknown@formal@none@1@S@Read the website before you do that, though, since not all programs offer terminal master's (none of the programs I applied to do).@@@@1@23@@danf@4-5-2012 222004210130@unknown@formal@none@1@S@Do your homework.@@@@1@3@@danf@4-5-2012 222004210140@unknown@formal@none@1@S@Professors are much, much busier than you are; if you demonstrate that you are too lazy to look things up on the Web, why should they spend time answering your email?@@@@1@31@@danf@4-5-2012 222004210150@unknown@formal@none@1@S@--- For past posts on graduate school and applying to graduate school, click here.@@@@1@14@@danf@4-5-2012 223000010010@unknown@formal@none@1@S@Welcome to the LingPipe blog@@@@1@5@@danf@4-5-2012 223000010020@unknown@formal@none@1@S@In anticipation of the 2.2 release of LingPipe, we decided to get with the program and create a blog.@@@@1@19@@danf@4-5-2012 223000130010@unknown@formal@none@1@S@Confidence-Based Gene Mentions for all of MEDLINE@@@@1@7@@danf@4-5-2012 223000130020@unknown@formal@none@1@S@I ran LingPipe’s new confidence-based named-entity extractor over every title and abstract body in MEDLINE.@@@@1@15@@danf@4-5-2012 223000130030@unknown@formal@none@1@S@The model is the one distributed on our site built from the NLM GeneTag corpus (a refined version of the first BioCreative corpus) — that’s a compiled .@@@@1@29@@danf@4-5-2012 223000130040@unknown@formal@none@1@S@There’s just a single category, .@@@@1@7@@danf@4-5-2012 223000130050@unknown@formal@none@1@S@The 2006 MEDLINE baseline contains 10.2 billion characters in titles and abstracts (with brackets for translations cut out of titles and truncation messages removed form abstracts).@@@@1@26@@danf@4-5-2012 223000130060@unknown@formal@none@1@S@I extracted the text using LingPipe’s MEDLINE parser and wrote the output in a gzipped form almost identical to that used in NLM’s GeneTag corpus (also used for BioCreative).@@@@1@29@@danf@4-5-2012 223000130070@unknown@formal@none@1@S@I set the minimum confidence to be 0.001.@@@@1@8@@danf@4-5-2012 223000130080@unknown@formal@none@1@S@I set the caches to be 10M entries each, but then capped the JVM memory at 2GB, so the soft references in the cache are getting collected when necessary.@@@@1@29@@danf@4-5-2012 223000130090@unknown@formal@none@1@S@I should try it with a smaller cache that won’t get GC-ed and see if the cache is better at managing itself than the GC is.@@@@1@26@@danf@4-5-2012 223000130100@unknown@formal@none@1@S@Including the I/O , XML parsing, gzipping and unzipping and MEDLINE DOM construction, it all ran over all of MEDLINE in just under 9 hours.@@@@1@25@@danf@4-5-2012 223000130110@unknown@formal@none@1@S@That’s 330,000 characters/second!!!@@@@1@3@@danf@4-5-2012 223000130120@unknown@formal@none@1@S@That’s on a fairly modest 1.8GHz dual opteron, 8GB PC2700 ECC memory, Windows64, 1.5 64-bit JDK in server mode).@@@@1@19@@danf@4-5-2012 223000130130@unknown@formal@none@1@S@That’s in a single analysis thread (of course, the 1.5 server JVM uses a separate thread for GC).@@@@1@18@@danf@4-5-2012 223000130140@unknown@formal@none@1@S@All I can say is woot!@@@@1@6@@danf@4-5-2012 223000140010@unknown@formal@none@1@S@Aho-Corasick Wikipedia Entry@@@@1@3@@danf@4-5-2012 223000140020@unknown@formal@none@1@S@I edited my first substantial Wikipedia page today:@@@@1@8@@danf@4-5-2012 223000140030@unknown@formal@none@1@S@Wikipedia: Aho-Corasick Algorithm@@@@1@3@@danf@4-5-2012 223000140040@unknown@formal@none@1@S@There’s been discussion of the Wikipedia’s accuracy ever since:@@@@1@9@@danf@4-5-2012 223000140050@unknown@formal@none@1@S@Nature article on Wikipedia.@@@@1@4@@danf@4-5-2012 223000140060@unknown@formal@none@1@S@There was even a New Yorker article this week.@@@@1@9@@danf@4-5-2012 223000140070@unknown@formal@none@1@S@Of course, The Onion said it best, in their article Wikipedia Celebrates 750 Years Of American Independence.@@@@1@17@@danf@4-5-2012 223000140080@unknown@formal@none@1@S@I wanted to link in some doc for the exact dictionary matching I just built for LingPipe 2.4 ( ), but I couldn’t find a good intro to the Aho-Corasick algorithm online.@@@@1@32@@danf@4-5-2012 223000140090@unknown@formal@none@1@S@The former Wikipedia article was confusing in its description of the data structure and wrong about the complexity bound (read in terms of number of entries versus size of dictionary strings).@@@@1@31@@danf@4-5-2012 223000140100@unknown@formal@none@1@S@I restated it in the usual manner (e.g., that used by Dan Gusfield in Algorithms on Strings, Trees and Sequences) .@@@@1@21@@danf@4-5-2012 223000140110@unknown@formal@none@1@S@And I provided an example like the one I’ve been using for unit testing.@@@@1@14@@danf@4-5-2012 223000140120@unknown@formal@none@1@S@The usual statement of Aho-Corasick is that it’s linear in dictionary size plus input plus number of outputs, as there may be quadratically many ouptuts.@@@@1@25@@danf@4-5-2012 223000140130@unknown@formal@none@1@S@Thinking a bit more deeply, it can’t really be quadratic without a quadratic sized dictionary.@@@@1@15@@danf@4-5-2012 223000140140@unknown@formal@none@1@S@For instance, with a dictionary {a, aa, aaa, aaaa, aaaaa}, there are quadratically many outputs for aaaaa, but the dictionary is quadratic in aaaaa (sum of first n integers: 5+4+3+2+1).@@@@1@30@@danf@4-5-2012 223000140150@unknown@formal@none@1@S@With a fixed dictionary, runtime is always linear in the input, though there may be outputs proportional to the number of dictionary entries for each input symbol.@@@@1@27@@danf@4-5-2012 223000140160@unknown@formal@none@1@S@I also restated it in terms of suffix trees rather than finite-state automata, as that matches the usual presentation.@@@@1@19@@danf@4-5-2012 223000520010@unknown@formal@none@1@S@Intro to IR book online@@@@1@5@@danf@4-5-2012 223000520020@unknown@formal@none@1@S@The following book promises to become the book on information retrieval.@@@@1@11@@danf@4-5-2012 223000520030@unknown@formal@none@1@S@Although there’s less on index compression than Witten et al.’s Managing Gigabytes, it is much broader and more up to date.@@@@1@21@@danf@4-5-2012 223000520040@unknown@formal@none@1@S@Chapters 13-18 aren’t really IR-specific at all, covering topics such as classification, clustering and latent semantic indexing.@@@@1@17@@danf@4-5-2012 223000520050@unknown@formal@none@1@S@That means this is a great place for an introduction to the way LingPipe does all these things, as we’ve followed standard practice in all of our models.@@@@1@28@@danf@4-5-2012 223000520060@unknown@formal@none@1@S@Here’s the reference, with a hotlink:@@@@1@6@@danf@4-5-2012 223000520070@unknown@formal@none@1@S@Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze.@@@@1@8@@danf@4-5-2012 223000520080@unknown@formal@none@1@S@(forthcoming) Introduction to Information Retrieval.@@@@1@5@@danf@4-5-2012 223000520090@unknown@formal@none@1@S@Cambridge University Press.@@@@1@3@@danf@4-5-2012 223000520100@unknown@formal@none@1@S@From what I’ve read, the treatments of various topics are better thought out and contain much more practical advice than the corresponding sections in Manning and Schütze’s previous book.@@@@1@29@@danf@4-5-2012 223000520110@unknown@formal@none@1@S@I don’t know how they’ve broken up the writing, but Prabhakar Raghavan, the third author, not only works at Yahoo! but is the editor-in-chief of the CS journal, the Journal of the ACM.@@@@1@33@@danf@4-5-2012 223000520130@unknown@formal@none@1@S@There’s still plenty of time to send the authors feedback and earn a coveted spot in the acknowledgements of a book destined to be widely read.@@@@1@26@@danf@4-5-2012 223000740010@unknown@formal@none@1@S@Feature Hash Code Collisions in Linear Classifiers@@@@1@7@@danf@4-5-2012 223000740020@unknown@formal@none@1@S@I (Bob) am starting to feel like a participant in an early 20th century epistolary academic exchange (e.g. that between Mr. Russell and Mr. Strawson).@@@@1@25@@danf@4-5-2012 223000740030@unknown@formal@none@1@S@In a comment to John Langford’s response to my blog entry recapitulating his comments after his talk, Kuzman Ganchev points out that he and Mark Dredze did the empirical legwork in their 2008 paper:@@@@1@34@@danf@4-5-2012 223000740040@unknown@formal@none@1@S@
  • Ganchev, Kuzman and Mark Dredze. 2008.@@@@1@6@@danf@4-5-2012 223000740060@unknown@formal@none@1@S@Small Statistical Models by Random Feature Mixing.@@@@1@7@@danf@4-5-2012 223000740070@unknown@formal@none@1@S@In Proceedings of the ACL-2008 Workshop on Mobile Language.
    • @@@@1@9@@danf@4-5-2012 223000740080@unknown@formal@none@1@S@To summarize, we’re considering the effect on classification accuracy of using hash codes (modulo some fixed n) of feature representations as dimensional identifiers rather than requiring a unique identifier for each of m underlying features.@@@@1@35@@danf@4-5-2012 223000740090@unknown@formal@none@1@S@The reason this is interesting is that it requires much less memory and far less computational effort to compute features this way.@@@@1@22@@danf@4-5-2012 223000740100@unknown@formal@none@1@S@The reason it might be problematic for accuracy is that there’s an increasing likeliood of collisions as the number of parameters n decreases.@@@@1@23@@danf@4-5-2012 223000740110@unknown@formal@none@1@S@For instance, character n-gram hash codes may be computed at a cost of only a couple assignments and arithmetic operations per character by the Karp-Rabin algorithm, which only implicitly represent the n-grams themselves.@@@@1@33@@danf@4-5-2012 223000740120@unknown@formal@none@1@S@For Latin1 (ISO-8859-1) encoded text, Karp-Rabin can even be computed online with binary input streams ( ) without the expense of decoding Unicode characters.@@@@1@24@@danf@4-5-2012 223000740130@unknown@formal@none@1@S@With n-gram-based features, input streams are usually effective with other character encodings.@@@@1@12@@danf@4-5-2012 223000740140@unknown@formal@none@1@S@More complex features may be handled by simple modifications of the Karp-Rabin algorithm.@@@@1@13@@danf@4-5-2012 223000740150@unknown@formal@none@1@S@Ganchev and Dredze showed that for many NLP problems (e.g. spam filtering, 20 newsgroups, Reuters topics, appliance reviews), there is very little loss from drastic reductions in n relative to m.@@@@1@31@@danf@4-5-2012 223000740160@unknown@formal@none@1@S@This is very good news indeed.@@@@1@6@@danf@4-5-2012 223000740170@unknown@formal@none@1@S@Getting back to John Langford’s last post, I would like to answer my own question about how having multiple hash codes per feature helps maintain discriminative power in the face of collisions.@@@@1@32@@danf@4-5-2012 223000740180@unknown@formal@none@1@S@It may seem counterintuive, as having more features (2 * m) for the same number of parameters (n) seems like there will simply be more collisions.@@@@1@26@@danf@4-5-2012 223000740190@unknown@formal@none@1@S@Let’s consider a very simple case where there is a feature f which is split into two features f0 and f1 without collision.@@@@1@23@@danf@4-5-2012 223000740200@unknown@formal@none@1@S@With maximum likelihood, if the original weight for f is β, then the weights for f0 and f1 and f2 will be β/2 (or any linear interpolation).@@@@1@27@@danf@4-5-2012 223000740210@unknown@formal@none@1@S@With Laplace priors (L1 regularization), the behavior is the same because the penalty is unchanged abs(β) = abs(β/2) + abs(β/2).@@@@1@20@@danf@4-5-2012 223000740220@unknown@formal@none@1@S@But, with Gaussian priors (L2 regularization), the penalty is no longer equivalent, because with β != 0, β2 > (β/2)2 + (β/2)2.@@@@1@22@@danf@4-5-2012 223000740230@unknown@formal@none@1@S@Setting aside regularization, let’s work through an example with two topics with the following generative model (adapted from Griffiths’ and Steyvers’ LDA paper):@@@@1@23@@danf@4-5-2012 223000740240@unknown@formal@none@1@S@So topic 0 is about geography and topic 1 about finance.@@@@1@11@@danf@4-5-2012 223000740250@unknown@formal@none@1@S@In topic 0, there is a 50% chance of generating the word "river", a 50% chance of generating the word "bank", and no chance of generating the word "loan".@@@@1@29@@danf@4-5-2012 223000740260@unknown@formal@none@1@S@It is easy to identify a set of regression parameters that has perfect classification behavior: β = (1,0,-1) [the maximum likelihood solution will not actually be identified; with priors, the scale or variance parameter of the prior determines the scale of the coefficients].@@@@1@43@@danf@4-5-2012 223000740270@unknown@formal@none@1@S@Now what happens when we blow out each feature to two features and allow collisions?@@@@1@15@@danf@4-5-2012 223000740280@unknown@formal@none@1@S@The generative model is the same, but each feature is replicated.@@@@1@11@@danf@4-5-2012 223000740290@unknown@formal@none@1@S@If there is a collision between the first code for "river" and the first code for "loan", the resulting coefficients look like:@@@@1@22@@danf@4-5-2012 223000740300@unknown@formal@none@1@S@The resulting coefficients again produce a perfect classifier.@@@@1@8@@danf@4-5-2012 223000740310@unknown@formal@none@1@S@The collision at code 0 is simply a non-discriminative feature and the split versions pick up the slack.@@@@1@18@@danf@4-5-2012 223000740320@unknown@formal@none@1@S@If the collision is between a discriminative and non-discriminative feature code, there is still a perfect set of coefficients:@@@@1@19@@danf@4-5-2012 223000740330@unknown@formal@none@1@S@Of course, real problems aren’t quite as neat as the examples, and as we pointed out, regularization is non-linear except for maximum likelihood and Laplace priors.@@@@1@26@@danf@4-5-2012 223000740340@unknown@formal@none@1@S@In the real world, we typically find "new" features at runtime (e.g. a character 8-gram that was never seen during training).@@@@1@21@@danf@4-5-2012 223000740350@unknown@formal@none@1@S@There is a very long tail for most linguistic processes.@@@@1@10@@danf@4-5-2012 223000740360@unknown@formal@none@1@S@Luckily, there is also a lot of redundancy in most classification problems.@@@@1@12@@danf@4-5-2012 223000740370@unknown@formal@none@1@S@This problem doesn’t occur in the hermetically sealed world of a fixed training and test set with feature pruning (as in the ECML KDD 2006 Spam Detection Challenge, which distributed data as bags of words with words occurring only if they occurred 4 or more times in the training data).@@@@1@50@@danf@4-5-2012 223001020010@unknown@formal@none@1@S@Scientific Innovator’s Dilemma@@@@1@3@@danf@4-5-2012 223001020020@unknown@formal@none@1@S@After some e-mail exchange with Mark Johnson about how to stimulate some far-out research that might be fun to read about, I was sitting at the dinner table with Alice Mitzi, ranting about the sociology of science.@@@@1@37@@danf@4-5-2012 223001020030@unknown@formal@none@1@S@My particular beef is low acceptance rates and the conservative nature of tenure committees, program committees, and grant review panels.@@@@1@20@@danf@4-5-2012 223001020040@unknown@formal@none@1@S@It makes it hard to get off the ground with a new idea, while making it far too easy to provide a minor, often useless, improvement on something already well known.@@@@1@31@@danf@4-5-2012 223001020050@unknown@formal@none@1@S@Part of the problem is that the known is just a lot easier to recognize and review.@@@@1@17@@danf@4-5-2012 223001020060@unknown@formal@none@1@S@I don’t spend days on reviews like I did as a grad student — if the writer can’t explain the main idea in the abstract/intro, into the reject pile it goes without my trying to work through all the math.@@@@1@40@@danf@4-5-2012 223001020070@unknown@formal@none@1@S@Mitzi listened patiently and after I eventually tailed off, said “Isn’t that just like the innovator’s dilemma, only for science?”.@@@@1@20@@danf@4-5-2012 223001020080@unknown@formal@none@1@S@Hmm, I thought, “hmm”, I mumbled, then my brain caught up and I finally let out an “a-ha”.@@@@1@18@@danf@4-5-2012 223001020090@unknown@formal@none@1@S@Then I said, “I should blog about this!”.@@@@1@8@@danf@4-5-2012 223001020100@unknown@formal@none@1@S@I learned about the problem in the title of one of the best business books I’ve ever read, The Innovator’s Dilemma, by Clayton M. Christensen.@@@@1@25@@danf@4-5-2012 223001020110@unknown@formal@none@1@S@It’s full of case studies about why players with the dominant positions in their industries fail.@@@@1@16@@danf@4-5-2012 223001020120@unknown@formal@none@1@S@You can read the first chapter and disk drives case study online, or cut to the dryer Wikipedia presentation of disruptive technology.@@@@1@22@@danf@4-5-2012 223001020130@unknown@formal@none@1@S@The basic dilemma is that an existing business generates so much revenue and at such a high margin, that any new activity not directly related to this existing business can’t be justified.@@@@1@32@@danf@4-5-2012 223001020140@unknown@formal@none@1@S@My favorite case study is of earth-movers.@@@@1@7@@danf@4-5-2012 223001020150@unknown@formal@none@1@S@Back in the day (but not too far back) we had steam shovels that used cables to move their enormous shovels.@@@@1@21@@danf@4-5-2012 223001020160@unknown@formal@none@1@S@They were big, and they moved lots of earth.@@@@1@9@@danf@4-5-2012 223001020170@unknown@formal@none@1@S@If you needed to do strip mining, foundation digging for skyscrapers, or needed to lay out a city’s road system, these were just what you wanted.@@@@1@26@@danf@4-5-2012 223001020180@unknown@formal@none@1@S@The more dirt they moved the better.@@@@1@7@@danf@4-5-2012 223001020190@unknown@formal@none@1@S@So along comes the gasoline powered engine.@@@@1@7@@danf@4-5-2012 223001020200@unknown@formal@none@1@S@The steam shovel companies looked at the new techology and quickly adopted it; swapping out steam for gasoline meant you could move more dirt with more or less the same set of cables.@@@@1@33@@danf@4-5-2012 223001020210@unknown@formal@none@1@S@It’s what we in the software business call a “no brainer”.@@@@1@11@@danf@4-5-2012 223001020220@unknown@formal@none@1@S@A few years later, an enterprising inventor figured out how to replace the cable actuators with hydraulics.@@@@1@17@@danf@4-5-2012 223001020230@unknown@formal@none@1@S@When first introduced, hydraulics were relatively weak compared to cables, so you couldn’t build a shovel big enough to compete with a cable-actuated gasoline-powered shovel.@@@@1@25@@danf@4-5-2012 223001020240@unknown@formal@none@1@S@The big shovel companies looked at hydraulics, but couldn’t figure out how to make money with them.@@@@1@17@@danf@4-5-2012 223001020250@unknown@formal@none@1@S@The first hydraulic shovels were tiny, being good only for jobs like digging the foundation for a house or digging a trench from a house to sewer mains.@@@@1@28@@danf@4-5-2012 223001020260@unknown@formal@none@1@S@Even more importantly, there was no existing market for small earth movers compared to the much more lucrative market for big earth movers, and even if you could capture all the little stuff, it still wouldn’t affect the big company’s bottom line.@@@@1@42@@danf@4-5-2012 223001020270@unknown@formal@none@1@S@So new companies sprung up in a new market to sell hydraulic shovels that could fit on a small truck.@@@@1@20@@danf@4-5-2012 223001020280@unknown@formal@none@1@S@As hydraulic technology continued to improve in strength, more and more markets opened up that took slightly more power.@@@@1@19@@danf@4-5-2012 223001020290@unknown@formal@none@1@S@Even so, nothing that’d make a dent in the bottom line of the big cable-activated shovel companies.@@@@1@17@@danf@4-5-2012 223001020300@unknown@formal@none@1@S@Eventually, hydraulics got powerful enough that they could compete with cable-activated shovels.@@@@1@12@@danf@4-5-2012 223001020310@unknown@formal@none@1@S@At this point, the cable-actuated shovel companies mainly went out of business.@@@@1@12@@danf@4-5-2012 223001020320@unknown@formal@none@1@S@Up until just before the capabilities crossed, it still didn’t make sense in terms of the big company’s bottom line to move to hydraulics.@@@@1@24@@danf@4-5-2012 223001020330@unknown@formal@none@1@S@There just wasn’t enough income in it.@@@@1@7@@danf@4-5-2012 223001020340@unknown@formal@none@1@S@Until too late.@@@@1@3@@danf@4-5-2012 223001020350@unknown@formal@none@1@S@Christensen’s book is loaded with case studies, and it’s easy to think of more once you have the pattern down.@@@@1@20@@danf@4-5-2012 223001020360@unknown@formal@none@1@S@The business types prefer generic, unscaled graphs like this one to illustrate what’s going on:@@@@1@15@@danf@4-5-2012 223001020370@unknown@formal@none@1@S@How disruptive technology gains a hold over time, by gradually moving into more lucrative markets (source: Wikipedia)@@@@1@17@@danf@4-5-2012 223001020380@unknown@formal@none@1@S@Smaller disks disrupted larger disks for the same reason; sure, they could fit into a minicomputer (or microcomputer), but they cost a lot per byte.@@@@1@25@@danf@4-5-2012 223001020390@unknown@formal@none@1@S@At every stage of disk diameter downsizing, the dominant players mostly went bankrupt or left the business in the face of the up-and-coming smaller disk manufactures, who always managed to overtake their big-disk competitors in terms of capacity, price (and reliability, if I recall correctly).@@@@1@45@@danf@4-5-2012 223001020400@unknown@formal@none@1@S@You’d think the big companies would have learned their lesson after the third iteration, but that just shows how strong a problem the innovator’s dilemma remains.@@@@1@26@@danf@4-5-2012 223001020410@unknown@formal@none@1@S@In computational linguistics and machine learning research, the big company on top is whatever technique has the best performance on some task.@@@@1@22@@danf@4-5-2012 223001020420@unknown@formal@none@1@S@I thought I’d never see the end of minor variants on three-state acoustic HMMs for context-dependent triphones in speech recognition when we knew they could never sort ‘d’ from ‘t’ (here’s some disruptive “landmark-based” speech recognition).@@@@1@36@@danf@4-5-2012 223001020430@unknown@formal@none@1@S@Disruptive technologies might not have state of the art performance or might not scale, but they should have some redeeming features.@@@@1@21@@danf@4-5-2012 223001020440@unknown@formal@none@1@S@One could view statistical NLP as being disruptive itself; in the beginning, it only did sequences with frequency-based estimators.@@@@1@19@@danf@4-5-2012 223001020450@unknown@formal@none@1@S@But remember, just because a technique performs below the best published results doesn’t make it disruptive.@@@@1@16@@danf@4-5-2012 223001020460@unknown@formal@none@1@S@The remaining dilemma is that none of the follow-on books by Christensen or others provide a good read, much less a solution to the innovator’s dilemma.@@@@1@26@@danf@4-5-2012 223001150010@unknown@formal@none@1@S@Artists Ship, or the Best is the Enemy of the Good@@@@1@11@@danf@4-5-2012 223001150020@unknown@formal@none@1@S@

    Artists Ship

    @@@@1@2@@danf@4-5-2012 223001150030@unknown@formal@none@1@S@I try not to just link to other articles in this blog, but I was extremely taken with Paul Graham’s blog post The other half of “artists ship”, because it rang so true to my experience as both a scientist and software engineer.@@@@1@43@@danf@4-5-2012 223001150040@unknown@formal@none@1@S@The expression is attributed to Steve Jobs in Steve Levy’s book on the Mac Insanely Great, which I’ve cut and pasted from Alex Golub’s blog:@@@@1@25@@danf@4-5-2012 223001150050@unknown@formal@none@1@S@… REAL ARTISTS SHIP …@@@@1@5@@danf@4-5-2012 223001150051@unknown@formal@none@1@S@One’s creation, quite simply, did not exist as art if it was not out there, available for consumption, doing well.@@@@1@20@@danf@4-5-2012 223001150060@unknown@formal@none@1@S@Was [Douglas] Engelbart an artist?@@@@1@5@@danf@4-5-2012 223001150070@unknown@formal@none@1@S@A prima donna — he didn’t ship.@@@@1@7@@danf@4-5-2012 223001150080@unknown@formal@none@1@S@What were the wizards of PARC?@@@@1@6@@danf@4-5-2012 223001150090@unknown@formal@none@1@S@Haughty aristocrats — they didn’t ship.@@@@1@6@@danf@4-5-2012 223001150100@unknown@formal@none@1@S@The final step of an artist — the single validating act — was getting his or her work into boxes … to make a difference in the world and a dent in the universe, you had to ship.@@@@1@38@@danf@4-5-2012 223001150110@unknown@formal@none@1@S@A while back, I commented on the artist-programmer connection in the blog post Industrialist or Auteur?, which pinned the blame on “obsessive pedantry”.@@@@1@23@@danf@4-5-2012 223001150120@unknown@formal@none@1@S@

    The Best is the Enemy of the Good

    @@@@1@8@@danf@4-5-2012 223001150130@unknown@formal@none@1@S@Two centuries and a decade earlier in 1772, Voltaire got to the bottom of why many of us have trouble finishing projects, stating Le mieux est l’ennemi du bien. (roughly “the best [better/perfect] is the enemy of the good”).@@@@1@39@@danf@4-5-2012 223001150140@unknown@formal@none@1@S@I remember seeing this first in the introduction to Barwise and Perry’s book Situations and Attitudes, where they thank their editor for reminding them that among the good qualities a book may possess, existence is a quite important one.@@@@1@39@@danf@4-5-2012 223001150150@unknown@formal@none@1@S@Barwise and Perry copped to falling prey to the temptation to keep tweaking something to make it better while never delivering anything.@@@@1@22@@danf@4-5-2012 223001150160@unknown@formal@none@1@S@That’s one reason why deadlines help, be they real (this morning’s NAACL/HLT submission deadline) or imaginary (if I don’t get this blog entry done today, no supper).@@@@1@27@@danf@4-5-2012 223001150170@unknown@formal@none@1@S@

    Build and Release

    @@@@1@3@@danf@4-5-2012 223001150180@unknown@formal@none@1@S@If you had to wait for my ultimate NLP API, LingPipe wouldn’t exist.@@@@1@13@@danf@4-5-2012 223001150190@unknown@formal@none@1@S@Almost every method in every class could use improvement in everything from unit tests to documentation to efficiency.@@@@1@18@@danf@4-5-2012 223001150200@unknown@formal@none@1@S@I have reasonably high standards (as the fans of extreme programming like to say, there are only two quality settings of interest to programmers, great and lives-depend-on-it), but I’m a pushover compared to what even a small (200-400 person, 30 or so of whom were core product coders) company like SpeechWorks required in the way of QA.@@@@1@57@@danf@4-5-2012 223001150210@unknown@formal@none@1@S@At SpeechWorks, I had proposed an API for language segmentation (breaking a document down into spans by language, primarily for European text-to-speech from e-mail apps), which was informed by a long (and good) marketing document and read by no fewer than a dozen commentators, who also read later drafts.@@@@1@49@@danf@4-5-2012 223001150220@unknown@formal@none@1@S@The code was written and reviewed by two of us, and we interfaced with the testing group who had to put the code through functional tests on a host of platforms.@@@@1@31@@danf@4-5-2012 223001150230@unknown@formal@none@1@S@And then there was release engineering.@@@@1@6@@danf@4-5-2012 223001150240@unknown@formal@none@1@S@Oh, and did I mention the documentation department?@@@@1@8@@danf@4-5-2012 223001150250@unknown@formal@none@1@S@My first pass at API design was a little too Ph.D.-centric (no, my commentators said, the users wouldn’t want to tune interpolation parameters for character language models at run time — just guess a good value for them, please); LingPipe is what happens when there’s no one from marketing commenting on the API!@@@@1@53@@danf@4-5-2012 223001150260@unknown@formal@none@1@S@If you increase your team size to something like Microsoft (see How MS Builds Software), you won’t even get through the hierarchical build process in the month it took SpeechWorks to roll out software, and then you’ll be waiting on the internationalization team to translate your deathless dialog box prose into dozens of language.@@@@1@54@@danf@4-5-2012 223001150270@unknown@formal@none@1@S@Perhaps that’s why Mark Chu-Carroll finds enough to hold his interest working on Google’s builds!@@@@1@15@@danf@4-5-2012 223001150280@unknown@formal@none@1@S@

    A Dissertation’s Just Practice

    @@@@1@4@@danf@4-5-2012 223001150290@unknown@formal@none@1@S@The worst case of the not-good-enough problem I’ve seen is in academia.@@@@1@12@@danf@4-5-2012 223001150300@unknown@formal@none@1@S@Students somehow try to pack as much as they can into a thesis.@@@@1@13@@danf@4-5-2012 223001150310@unknown@formal@none@1@S@I sure did.@@@@1@3@@danf@4-5-2012 223001150320@unknown@formal@none@1@S@I had seven chapters outlined, and after five, my advisor (Ewan Klein) told me to stop, I had enough.@@@@1@19@@danf@4-5-2012 223001150330@unknown@formal@none@1@S@Of course, some advisors never think their students have done enough work — that’s just the same problem from management’s perspective.@@@@1@21@@danf@4-5-2012 223001150340@unknown@formal@none@1@S@My own advice to students was to save their life’s work for the rest of their life — a dissertation’s just practice.@@@@1@22@@danf@4-5-2012 223001150350@unknown@formal@none@1@S@As evidence of that, they’re graded mostly on form, not content.@@@@1@11@@danf@4-5-2012 223001150360@unknown@formal@none@1@S@

    Revise and Resubmit

    @@@@1@3@@danf@4-5-2012 223001150370@unknown@formal@none@1@S@The Computational Lingusitics journal editorial board faces the same problem.@@@@1@10@@danf@4-5-2012 223001150380@unknown@formal@none@1@S@Robert Dale (the editor) found that most authors who were asked to revise and resubmit their paper (that is, not rejected or accepted outright) never got around to it.@@@@1@29@@danf@4-5-2012 223001150390@unknown@formal@none@1@S@Robert tracked some of the authors down and they said they simply didn’t have enough time to run all the extra experiments and analyses proposed by reviewers.@@@@1@27@@danf@4-5-2012 223001150400@unknown@formal@none@1@S@Robert asked us to rethink the way we came to conclusions, and instead of asking “could this be better?” ask “is it good enough to be interesting?”.@@@@1@27@@danf@4-5-2012 223001150410@unknown@formal@none@1@S@I couldn’t agree more.@@@@1@4@@danf@4-5-2012 223001430010@unknown@formal@none@1@S@Provost, Fawcett & Kohavi (1998) The Case Against Accuracy Estimation for Comparing Induction Algorithms@@@@1@14@@danf@4-5-2012 223001430020@unknown@formal@none@1@S@I couldn’t agree more with the first conclusion:@@@@1@8@@danf@4-5-2012 223001430030@unknown@formal@none@1@S@First, the justifications for using accuracy to compare classifiers are questionable at best.@@@@1@13@@danf@4-5-2012 223001430040@unknown@formal@none@1@S@of this paper:@@@@1@3@@danf@4-5-2012 223001430050@unknown@formal@none@1@S@
  • Foster Provost, Tom Fawcett, and Ron Kohavi. 1998.@@@@1@9@@danf@4-5-2012 223001430070@unknown@formal@none@1@S@The Case Against Accuracy Estimation for Comparing Induction Algorithms.@@@@1@9@@danf@4-5-2012 223001430080@unknown@formal@none@1@S@In ICML.
    • @@@@1@2@@danf@4-5-2012 223001430090@unknown@formal@none@1@S@In fact, I’d extend it to micro-averaged and macro-averaged F-measures, AUC, BEP, etc.,@@@@1@13@@danf@4-5-2012 223001430100@unknown@formal@none@1@S@Foster and crew’s argument is simple.@@@@1@6@@danf@4-5-2012 223001430110@unknown@formal@none@1@S@They evaluate naive Bayes, decision trees, boosted decision trees, and k-nearest neighbor algorithms on a handful of UCI machine learning repository problems.@@@@1@22@@danf@4-5-2012 223001430120@unknown@formal@none@1@S@They show that there aren’t what they call dominating ROC curves for any of the classifiers on any of the problems.@@@@1@21@@danf@4-5-2012 223001430130@unknown@formal@none@1@S@For example, here’s their figure 1 (they later apply smoothing to better estimate ROC curves):@@@@1@15@@danf@4-5-2012 223001430140@unknown@formal@none@1@S@The upshot is that depending on whether you need high recall or high precision, the “best” classifier is different.@@@@1@19@@danf@4-5-2012 223001430150@unknown@formal@none@1@S@As I’ve said before, it’s horses for courses.@@@@1@8@@danf@4-5-2012 223001430160@unknown@formal@none@1@S@To be a little more specific, they plot receiver operating characteristic (ROC) curves for the classifiers, which shows (1-specificity) versus sensitivity.@@@@1@21@@danf@4-5-2012 223001430170@unknown@formal@none@1@S@
  • sensitivity = truePos / (truePos + falseNeg)
    • @@@@1@8@@danf@4-5-2012 223001430180@unknown@formal@none@1@S@
  • specificity = trueNeg / (trueNeg + falsePos)
    • @@@@1@8@@danf@4-5-2012 223001430190@unknown@formal@none@1@S@In LingPipe, any ranked classifier can be evaluated for ROC curve using the method:@@@@1@14@@danf@4-5-2012 223001430200@unknown@formal@none@1@S@It’d be nice to see this work extended to today’s most popular classifiers: SVMs and logistic regression.@@@@1@17@@danf@4-5-2012 223002640010@unknown@formal@none@1@S@The Long Road to CRFs@@@@1@5@@danf@4-5-2012 223002640020@unknown@formal@none@1@S@

    CRFs are Done

    @@@@1@3@@danf@4-5-2012 223002640030@unknown@formal@none@1@S@The first bit of good news is that LingPipe 3.9 is within days of release.@@@@1@15@@danf@4-5-2012 223002640040@unknown@formal@none@1@S@CRFs are coded, documented, unit tested, and I’ve even written a long-ish tutorial with hello-world examples for tagging and chunking, and a longer example of chunking with complex features evaluated over:@@@@1@31@@danf@4-5-2012 223002640050@unknown@formal@none@1@S@
  • CoNLL 2003 Named Entity Bakeoff@@@@1@5@@danf@4-5-2012 223002640060@unknown@formal@none@1@S@

    And They’re Speedy

    @@@@1@3@@danf@4-5-2012 223002640070@unknown@formal@none@1@S@The second bit of good news is that it looks like we have near state-of-the-art performance in terms of speed.@@@@1@20@@danf@4-5-2012 223002640080@unknown@formal@none@1@S@It’s always hard to compare systems without exactly recreating the feature extractors, requirements for convergence, hardware setup and load, and so on.@@@@1@22@@danf@4-5-2012 223002640090@unknown@formal@none@1@S@I was looking at
  • Naoaki Okazaki’s CRFsuite Benchmarks
    • for comparison.@@@@1@10@@danf@4-5-2012 223002640120@unknown@formal@none@1@S@Okazaki also evaluated first-order chain CRFs, though on the CoNLL 2000 English phrase chunking data, which has fewer tags than the CoNLL 2003 English named entity data.@@@@1@27@@danf@4-5-2012 223002640130@unknown@formal@none@1@S@While my estimator may be a tad slower (it took about 10s/epoch during stochastic gradient), I’m applying priors, and I think the tag set is a bit bigger.@@@@1@28@@danf@4-5-2012 223002640140@unknown@formal@none@1@S@(Of course, if you use IO encoding rather than BIO encoding, like the Stanford named entity effort, then there’d be even fewer tags; on the other hands, if I followed Turian et al. (ref below), or the way we handle HMM encoding, there’d be more.)@@@@1@45@@danf@4-5-2012 223002640150@unknown@formal@none@1@S@It also looks like our run time is faster than the other systems benchmarked if you don’t consider feature extraction time (and I don’t think they did in the eval, but I may be wrong).@@@@1@35@@danf@4-5-2012 223002640160@unknown@formal@none@1@S@It’s running at 70K tokens/second for full forward-backward decoding; first-best would be faster.@@@@1@13@@danf@4-5-2012 223002640170@unknown@formal@none@1@S@

    All the Interfaces, Please

    @@@@1@4@@danf@4-5-2012 223002640180@unknown@formal@none@1@S@Like for HMMs, I implemented first-best, n-best with conditional probabilities, and a full forward-backward confidence evaluation.@@@@1@16@@danf@4-5-2012 223002640190@unknown@formal@none@1@S@For taggers, confidence is per tag per token; for chunkers, it’s per chunk.@@@@1@13@@danf@4-5-2012 223002640200@unknown@formal@none@1@S@

    Final Improvements

    @@@@1@2@@danf@4-5-2012 223002640210@unknown@formal@none@1@S@Yesterday, I was despairing a bit over how slow my approach was.@@@@1@12@@danf@4-5-2012 223002640220@unknown@formal@none@1@S@Then I looked at my code, instrumented the time spent in each component, and had my D’oh! moment(s).@@@@1@18@@danf@4-5-2012 223002640230@unknown@formal@none@1@S@

    Blocked Prior Updates

    @@@@1@3@@danf@4-5-2012 223002640240@unknown@formal@none@1@S@The first problem was that I was doing dense, stochastic prior updates.@@@@1@12@@danf@4-5-2012 223002640250@unknown@formal@none@1@S@That is, for every instance, I walked over the entire set of dense coefficient vectors, calculated the gradient, and applied it.@@@@1@21@@danf@4-5-2012 223002640260@unknown@formal@none@1@S@This was dominating estimation time.@@@@1@5@@danf@4-5-2012 223002640270@unknown@formal@none@1@S@So I applied a blocking strategy whereby the prior gradient is only applied every so often (say, every 100 instances).@@@@1@20@@danf@4-5-2012 223002640280@unknown@formal@none@1@S@This is the strategy discussed in Langford, Li and Zhang’s truncated gradient paper.@@@@1@13@@danf@4-5-2012 223002640290@unknown@formal@none@1@S@I can vouch for the fact that result vectors didn’t change much, and speed was hugely improved to the point where the priors weren’t taking much of the estimation time at all.@@@@1@32@@danf@4-5-2012 223002640300@unknown@formal@none@1@S@

    Caching Features

    @@@@1@2@@danf@4-5-2012 223002640310@unknown@formal@none@1@S@The other shortcoming of my initial implementation was that I was extracting features online rather than extracting them all into a cache.@@@@1@22@@danf@4-5-2012 223002640320@unknown@formal@none@1@S@After cleaning up the prior, feature extraction was taking orders of magnitude longer than everything else.@@@@1@16@@danf@4-5-2012 223002640330@unknown@formal@none@1@S@So I built a cache, and added yet another parameter to control whether to use it or not.@@@@1@18@@danf@4-5-2012 223002640340@unknown@formal@none@1@S@With the cache and rich feature extractors, the estimator needs 2GB; with online feature extraction, it’s about 20 times slower, but only requires around 300MB of memory or less.@@@@1@29@@danf@4-5-2012 223002640350@unknown@formal@none@1@S@

    Bug Fixes

    @@@@1@2@@danf@4-5-2012 223002640360@unknown@formal@none@1@S@There were several subtle and not-so-subtle bugs that needed to be fixed along the way.@@@@1@15@@danf@4-5-2012 223002640370@unknown@formal@none@1@S@One problem was that I forgot to scale the priors based on the number of training instances during estimation.@@@@1@19@@danf@4-5-2012 223002640380@unknown@formal@none@1@S@This led to huge over-regularization.@@@@1@5@@danf@4-5-2012 223002640390@unknown@formal@none@1@S@When I fixed this problem, the results started looking way better.@@@@1@11@@danf@4-5-2012 223002640400@unknown@formal@none@1@S@

    Structural Zeros

    @@@@1@2@@danf@4-5-2012 223002640410@unknown@formal@none@1@S@Another bug-like problem I had is that when decoding CRFs for chunkers, I needed to rule out certain illegal tag sequences.@@@@1@21@@danf@4-5-2012 223002640420@unknown@formal@none@1@S@The codec I abstracted to handle the encoding of chunkers and taggers and subsequent decoding could compute legal tag sequences and consistency with tokenizers, but the CRF itself couldn’t.@@@@1@29@@danf@4-5-2012 223002640421@unknown@formal@none@1@S@So I was getting illegal tag sequences out that caused the codec to crash.@@@@1@14@@danf@4-5-2012 223002640430@unknown@formal@none@1@S@So I built in structural zeros.@@@@1@6@@danf@4-5-2012 223002640440@unknown@formal@none@1@S@The simplest way to do it seemed to be to add a flag that only allowed tag transitions seen in the training data.@@@@1@23@@danf@4-5-2012 223002640450@unknown@formal@none@1@S@This way, I could enforce legal start tags, legal end tags, and legal transitions.@@@@1@14@@danf@4-5-2012 223002640460@unknown@formal@none@1@S@This had the nice side benefit of speeding things up, because I could skip calculations for structural zeros.@@@@1@18@@danf@4-5-2012 223002640470@unknown@formal@none@1@S@(This is one of the reasons Thorsten Brants’ TnT is so fast — it also applies this strategy to tags, only allowing tags seen in training data for given tokens.)@@@@1@30@@danf@4-5-2012 223002640480@unknown@formal@none@1@S@

    Feature Extraction Encapsulation

    @@@@1@3@@danf@4-5-2012 223002640490@unknown@formal@none@1@S@I was almost ready to go a couple of days ago.@@@@1@11@@danf@4-5-2012 223002640500@unknown@formal@none@1@S@But then I tried to build a richer feature extractor for the CoNLL entity data that used part-of-speech tags, token shape features, contextual features, prefixes and suffixes, etc.@@@@1@28@@danf@4-5-2012 223002640510@unknown@formal@none@1@S@Basically the “baseline” features suggested in Turian, Ratinov, Bengio and Roth’s survey of cluster features (more to come on that paper).@@@@1@21@@danf@4-5-2012 223002640520@unknown@formal@none@1@S@It turns out that the basic node and edge feature extractors, as I proposed almost six months ago, weren’t quite up to the job.@@@@1@24@@danf@4-5-2012 223002640530@unknown@formal@none@1@S@So I raised the abstraction level so that the features for a whole input were encapsulated in a features object that was called lazily by the decoders and/or estimator.@@@@1@29@@danf@4-5-2012 223002640540@unknown@formal@none@1@S@This allowed things like part-of-speech taggings to be computed once and then cached.@@@@1@13@@danf@4-5-2012 223002640550@unknown@formal@none@1@S@This will also allow online document features (like previous tagging decisions) to be rolled into the feature extractor, though it’ll take some work.@@@@1@23@@danf@4-5-2012 223002640560@unknown@formal@none@1@S@

    And a Whole Lotta’ Interfaces and Retrofitting

    @@@@1@7@@danf@4-5-2012 223002640570@unknown@formal@none@1@S@I added a whole new package, , to characterize the output of a first-best, n-best, and marginal tag probability tagger.@@@@1@21@@danf@4-5-2012 223002640580@unknown@formal@none@1@S@I then implemented these with CRFs and retrofitted them for HMMs.@@@@1@11@@danf@4-5-2012 223002640590@unknown@formal@none@1@S@I also pulled out the evaluator and generalized it.@@@@1@9@@danf@4-5-2012 223002640600@unknown@formal@none@1@S@Along the way, I deprecated a few interfaces, like , which is no longer needed given .@@@@1@19@@danf@4-5-2012 223002640610@unknown@formal@none@1@S@

    Still No Templated Feature Extraction

    @@@@1@5@@danf@4-5-2012 223002640620@unknown@formal@none@1@S@Looking at other CRF implementations, and talking to others who’d used them, I see that designing a language to specify feature extractions is popular.@@@@1@24@@danf@4-5-2012 223002640630@unknown@formal@none@1@S@Like other decisions in LingPipe, I’ve stuck to code-based solutions.@@@@1@10@@danf@4-5-2012 223002640640@unknown@formal@none@1@S@The problem with this is that it limits our users to Java developers.@@@@1@13@@danf@4-5-2012 224000370010@unknown@formal@none@1@S@Search-based Structured Prediction@@@@1@3@@danf@4-5-2012 224000370020@unknown@formal@none@1@S@I normally would not make a post such as this one (my blog is not my advertisementsphere), but given that it is unlikely this paper will appear in a conference in the near future (and conferences are just ads), I decided to include a link.@@@@1@45@@danf@4-5-2012 224000370030@unknown@formal@none@1@S@John, Daniel and I have been working on an algorithm called Searn for solving structured prediction problems.@@@@1@17@@danf@4-5-2012 224000370040@unknown@formal@none@1@S@I believe that it will be useful to NLP people, so I hope this post deserves the small space it takes up.@@@@1@22@@danf@4-5-2012 224000480010@unknown@formal@none@1@S@Approximating the Problem or the Solution@@@@1@6@@danf@4-5-2012 224000480020@unknown@formal@none@1@S@A while back I came across a paper that (in a completely separate context) argues for approximating problems in lieu of approximating solutions.@@@@1@23@@danf@4-5-2012 224000480030@unknown@formal@none@1@S@This idea has a nice analogue in NLP: should we (A) choose a simple model for which we can do exact inference or (B) choose a complex model that is closer to the truth for which exact inference is not tractable.@@@@1@41@@danf@4-5-2012 224000480040@unknown@formal@none@1@S@(A) is approximating the problem while (B) is approximating the solution.@@@@1@11@@danf@4-5-2012 224000480050@unknown@formal@none@1@S@It seems that all signs point to (B).@@@@1@8@@danf@4-5-2012 224000480060@unknown@formal@none@1@S@In almost every interesting case I know of, it helps (or at the very least doesn't hurt) to move to more complex models that are more expressive, even if this renders learning or search intractable.@@@@1@35@@danf@4-5-2012 224000480070@unknown@formal@none@1@S@This story is well known in word alignment (eg, GIZA) and MT (eg, model 4 decoding), but also has simpler examples in parsing (cf, McDonald), sequence labeling (cf, Sutton), relation extraction (cf, Culotta), as well as pretty much any area in which "joint inference" has been shown to be helpful.@@@@1@50@@danf@4-5-2012 224000480080@unknown@formal@none@1@S@One sobering example here is the story in word alignment, where one cannot go and directly use, say, model 4 for computing alignments, but must first follow a strict recipe: run a few iterations of model 1, followed by a few model 2, followed by some HMM, then model 4 (skipping model 3 all together). @@@@1@55@@danf@4-5-2012 224000480081@unknown@formal@none@1@S@The problem here is that learning model 4 parameters directly falls into local minima too easily, so one must initialize intelligently, by using outputs of previous iterations. @@@@1@27@@danf@4-5-2012 224000480082@unknown@formal@none@1@@My guess is that this result will continue to hold for training (though perhaps not predicting) with more an more complex models. @@@@0@22@@danf@4-5-2012 224000480083@unknown@formal@none@1@S@This is unfortunate, and there may be ways of coming up with learning algorithms that automatically initialize themselves by some mechanism for simplifying their own structure (seems like a fun open question, somewhat related to recent work by Smith).@@@@1@39@@danf@4-5-2012 224000480090@unknown@formal@none@1@S@Aside from a strong suggestion as to how to design models and inference procedure (i.e., ignore tractability in favor of expressiveness), there may be something interesting to say here about human language processing. @@@@1@33@@danf@4-5-2012 224000480091@unknown@formal@none@1@S@If it is indeed true that, for the most part, we can computationally move to more complex models, forgoing tractable search, then it is not implausible to imagine that perhaps humans do the same thing. @@@@1@35@@danf@4-5-2012 224000480092@unknown@formal@none@1@S@My knowledge in this area is sparse, but my general understanding is that various models of human language processing are disfavored because they would be too computationally difficult. @@@@1@28@@danf@4-5-2012 224000480093@unknown@formal@none@1@S@But if, as in old-school AI, we believe that humans just have a really good innate search algorithm, then this observation might lead us to believe that we have, ourselves, very complex, intractable "models" in our heads.@@@@1@37@@danf@4-5-2012 224000980010@unknown@formal@none@1@S@Reproducible Results@@@@1@2@@danf@4-5-2012 224000980020@unknown@formal@none@1@S@In an ideal world, it would be possible to read a paper, go out and implement the proposed algorithm, and obtain the same results.@@@@1@24@@danf@4-5-2012 224000980030@unknown@formal@none@1@S@In the real world, this isn't possible.@@@@1@7@@danf@4-5-2012 224000980040@unknown@formal@none@1@S@For one, if by "paper" we mean "conference paper," there's often just not enough space to spell out all the details.@@@@1@21@@danf@4-5-2012 224000980050@unknown@formal@none@1@S@Even how you do tokenization can make a big difference!@@@@1@10@@danf@4-5-2012 224000980060@unknown@formal@none@1@S@It seems reasonable that there should be sufficient detail in a journal paper to achieve essentially the same results, since there's (at least officially) not a space issue.@@@@1@28@@danf@4-5-2012 224000980070@unknown@formal@none@1@S@On the other hand, no one really publishes in journals in our subfamily of CS.@@@@1@15@@danf@4-5-2012 224000980080@unknown@formal@none@1@S@The next thing one can do is to release the software associated with a paper.@@@@1@15@@danf@4-5-2012 224000980090@unknown@formal@none@1@S@I've tried to do this in a handful of cases, but it can be a non-trivial exercise.@@@@1@17@@danf@4-5-2012 224000980100@unknown@formal@none@1@S@There are a few problems.@@@@1@5@@danf@4-5-2012 224000980110@unknown@formal@none@1@S@First, there's the question of how polished the software you put out should be.@@@@1@14@@danf@4-5-2012 224000980120@unknown@formal@none@1@S@Probably my most polished is megam (for learning classifiers) and the least polished is DPsearch (code from my AI stats paper).@@@@1@21@@danf@4-5-2012 224000980130@unknown@formal@none@1@S@It was a very nontrivial amount of effort to write up all the docs for megam and so on.@@@@1@19@@danf@4-5-2012 224000980140@unknown@formal@none@1@S@As a result, I hope that people can use it.@@@@1@10@@danf@4-5-2012 224000980150@unknown@formal@none@1@S@I have less hope for DPsearch --- you'd really have to know what you're doing to rip the guts out of it.@@@@1@22@@danf@4-5-2012 224000980160@unknown@formal@none@1@S@Nevertheless, I have occasionally received copies of code like my DPsearch from other people (i.e., unpolished code) and have still been able to use them successfully, albeit only for ML stuff, not for NLP stuff.@@@@1@35@@danf@4-5-2012 224000980170@unknown@formal@none@1@@ML stuff is nice because, for the most part, its self-contained.@@@@0@11@@danf@4-5-2012 224000980180@unknown@formal@none@1@S@NLP stuff often isn't: first you run a parser, then you have to have wordnet installed, then you have to have 100MB of data files, then you have to run scripts X, Y and Z before you can finally run the program.@@@@1@42@@danf@4-5-2012 224000980190@unknown@formal@none@1@S@The work I did for my thesis is a perfect example of this: instead of building all the important features into the main body of code I wrote, about half of them were implemented as Perl scripts that would essentially add "columns" to a CoNLL-style input format.@@@@1@47@@danf@4-5-2012 224000980200@unknown@formal@none@1@S@At the end, the input was like 25-30 columns wide, and if any were missing or out of order, bad things would happen.@@@@1@23@@danf@4-5-2012 224000980210@unknown@formal@none@1@S@As a result, it's a completely nontrivial exercise for me to release this beast.@@@@1@14@@danf@4-5-2012 224000980220@unknown@formal@none@1@S@The only real conceivable option would be to remove the non-important scripts, get the important ones back into the real code, and then release that.@@@@1@25@@danf@4-5-2012 224000980230@unknown@formal@none@1@S@But then there's no way the results would match exactly those from the paper/thesis.@@@@1@14@@danf@4-5-2012 224000980240@unknown@formal@none@1@S@I don't know of a solution to this problem.@@@@1@9@@danf@4-5-2012 224000980250@unknown@formal@none@1@S@I suppose it depends on what your goal is.@@@@1@9@@danf@4-5-2012 224000980260@unknown@formal@none@1@S@One goal is just to figure out some implementation details so that you can use them yourself.@@@@1@17@@danf@4-5-2012 224000980270@unknown@formal@none@1@S@For this, it would be perfectly acceptable in, say, my thesis situation, to just put up the code (perhaps the scripts too) and leave it at that.@@@@1@27@@danf@4-5-2012 224000980280@unknown@formal@none@1@S@There would be an implicit contract that you couldn't really expect too much from it (i.e., you shouldn't expect to run it).@@@@1@22@@danf@4-5-2012 224000980290@unknown@formal@none@1@S@A second goal is to use someone else's code as a baseline system to compare against.@@@@1@16@@danf@4-5-2012 224000980300@unknown@formal@none@1@S@This goal is lessened when common data is available, because you can compare to published results.@@@@1@16@@danf@4-5-2012 224000980310@unknown@formal@none@1@S@But often you don't care about the common data and really want to see how it works on other data.@@@@1@20@@danf@4-5-2012 224000980320@unknown@formal@none@1@S@Or you want to qualitatively compare your output to a baseline.@@@@1@11@@danf@4-5-2012 224000980330@unknown@formal@none@1@S@This seems harder to deal with.@@@@1@6@@danf@4-5-2012 224000980340@unknown@formal@none@1@S@If code goes up to solve this problem, it needs to be runnable.@@@@1@13@@danf@4-5-2012 224000980350@unknown@formal@none@1@S@And it needs to achieve pretty much the same results as published, otherwise funny things happen ("so and so reported scores of X but we were only able to achieve Y using their code", where Y < X).@@@@1@38@@danf@4-5-2012 224000980360@unknown@formal@none@1@S@This looks bad, but is actually quite understandable in many cases.@@@@1@11@@danf@4-5-2012 224000980370@unknown@formal@none@1@S@Maybe the solution here is, modulo copyright restrictions and licensing problems (ahem, LDC), just put up you models output as well.@@@@1@21@@danf@4-5-2012 224000980380@unknown@formal@none@1@S@This doesn't solve the direct problem, but maybe helps a bit.@@@@1@11@@danf@4-5-2012 224000980390@unknown@formal@none@1@@It also lets people see where you model screws up, so they can attempt to fix those problems.@@@@0@18@@danf@4-5-2012 224001510010@unknown@formal@none@1@S@ICML/UAI/COLT Workshops Posted@@@@1@3@@danf@4-5-2012 224001510020@unknown@formal@none@1@S@See here for the current list.@@@@1@6@@danf@4-5-2012 224001510030@unknown@formal@none@1@S@They include: Nonparametric Bayes (woohoo!), machine learning and music, Bayesian modeling applications, prior knowledge for text and language processing, sparse optimization and variable selection, as well as stand-alone workshops on the reinforcement learning competition and mining and learning with graphs.@@@@1@40@@danf@4-5-2012 224001510040@unknown@formal@none@1@S@Because I'm one of the organizers, I'd like to call attention to the Prior knowledge for text and language processing workshop.@@@@1@21@@danf@4-5-2012 224001510050@unknown@formal@none@1@S@We'd definitely like submissions on any of the following topics:@@@@1@10@@danf@4-5-2012 224001510060@unknown@formal@none@1@S@
  • Prior knowledge for language modeling, parsing, translation
    • @@@@1@7@@danf@4-5-2012 224001510070@unknown@formal@none@1@S@
  • Topic modeling for document analysis and retrieval
    • @@@@1@7@@danf@4-5-2012 224001510080@unknown@formal@none@1@S@
  • Parametric and non-parametric Bayesian models in NLP
    • @@@@1@7@@danf@4-5-2012 224001510090@unknown@formal@none@1@S@
  • Graphical models embodying structural knowledge of texts
    • @@@@1@7@@danf@4-5-2012 224001510100@unknown@formal@none@1@S@
  • Complex features/kernels that incorporate linguistic knowledge; kernels built from generative models
    • @@@@1@11@@danf@4-5-2012 224001510110@unknown@formal@none@1@S@
  • Limitations of purely data-driven learning techniques for text and language applications; performance gains due to incorporation of prior knowledge
    • @@@@1@19@@danf@4-5-2012 224001510120@unknown@formal@none@1@S@
  • Typology of different forms of prior knowledge for NLP (knowledge embodied in generative Bayesian models, in MDL models, in ILP/logical models, in linguistic features, in representational frameworks, in grammatical rules…)
    • @@@@1@30@@danf@4-5-2012 224001510130@unknown@formal@none@1@S@
  • Formal principles for combining rule-based and data-based approaches to NLP
    • @@@@1@10@@danf@4-5-2012 224001510140@unknown@formal@none@1@S@
  • Linguistic science and cognitive models as sources of prior knowledge
    • @@@@1@10@@danf@4-5-2012 224001510150@unknown@formal@none@1@S@Yes, I know that's a shameless plug, but do you really expect better from me?!@@@@1@15@@danf@4-5-2012 224001540010@unknown@formal@none@1@S@More complaining about automatic evaluation@@@@1@5@@danf@4-5-2012 224001540020@unknown@formal@none@1@S@I remember a few years ago complaining about automatic evaluation at conference was the thing to do.@@@@1@17@@danf@4-5-2012 224001540030@unknown@formal@none@1@S@(Ironically, so was writing papers about automatic evaluation!)@@@@1@8@@danf@4-5-2012 224001540040@unknown@formal@none@1@S@Things are saner now on both sides.@@@@1@7@@danf@4-5-2012 224001540050@unknown@formal@none@1@S@While what I'm writing here is interpretable as a gripe, it's really intended as a "did anyone else notice this" because it's somewhat subtle.@@@@1@24@@danf@4-5-2012 224001540060@unknown@formal@none@1@S@The evaluation metric I care about is Rouge, designed for summarization.@@@@1@11@@danf@4-5-2012 224001540070@unknown@formal@none@1@S@The primary difference between Rouge and Bleu is that Rouge is recall-oriented while Bleu is precision-oriented.@@@@1@16@@danf@4-5-2012 224001540080@unknown@formal@none@1@S@The way Rouge works is as follows.@@@@1@7@@danf@4-5-2012 224001540090@unknown@formal@none@1@S@Pick an ngram size.@@@@1@4@@danf@4-5-2012 224001540100@unknown@formal@none@1@S@Get a single system summary H and a single reference summary R (we'll get to multiple references shortly).@@@@1@18@@danf@4-5-2012 224001540110@unknown@formal@none@1@@Let |H| denote the size of bag the defined by H and let |H^R| denote the bag intersection.@@@@0@18@@danf@4-5-2012 224001540120@unknown@formal@none@1@S@Namely, the number of times some n-gram is allowed to appear in H^R is the min of the number of times it appears in H and R.@@@@1@27@@danf@4-5-2012 224001540130@unknown@formal@none@1@S@Take this number and divide by |R|.@@@@1@7@@danf@4-5-2012 224001540140@unknown@formal@none@1@S@This is the ngram recall for our system on this one example.@@@@1@12@@danf@4-5-2012 224001540150@unknown@formal@none@1@S@To extend this to more than one summary, we simple average the Rouges at each individual summary.@@@@1@17@@danf@4-5-2012 224001540160@unknown@formal@none@1@S@Now, suppose we have multiple references, R_1, R_2, ..., R_K.@@@@1@10@@danf@4-5-2012 224001540170@unknown@formal@none@1@S@In the original Rouge papers and implementation, we compute the score for a single sentence as the max over the references of the Rouge on that individual reference.@@@@1@28@@danf@4-5-2012 224001540180@unknown@formal@none@1@S@In other words, our score is the score against a single reference, where that reference is chosen optimistically.@@@@1@18@@danf@4-5-2012 224001540190@unknown@formal@none@1@@In later Rouge paper and implementation, this changed.@@@@0@8@@danf@4-5-2012 224001540200@unknown@formal@none@1@S@In the single-reference case, our score was |H^R|/|R|.@@@@1@8@@danf@4-5-2012 224001540210@unknown@formal@none@1@S@In the multiple reference setting, it is |H^(R_1 + R_2 + ... + R_K)|/|R_1 + R_2 + ... + R_K|, where + denotes bag union.@@@@1@25@@danf@4-5-2012 224001540220@unknown@formal@none@1@S@Apparently this makes the evaluation more stable.@@@@1@7@@danf@4-5-2012 224001540230@unknown@formal@none@1@S@(As an aside, there is no notion of a "too long" penalty because all system output is capped at some fixed length, eg., 100 words.)@@@@1@25@@danf@4-5-2012 224001540240@unknown@formal@none@1@S@Enough about how Rouge works.@@@@1@5@@danf@4-5-2012 224001540250@unknown@formal@none@1@S@Let's talk about how my DUC summarization system worked back in 2006.@@@@1@12@@danf@4-5-2012 224001540260@unknown@formal@none@1@S@First, we run BayeSum to get a score for each sentence.@@@@1@11@@danf@4-5-2012 224001540270@unknown@formal@none@1@S@Then, based on the score and about 10 other features, we perform sentence extraction, optimized against Rouge.@@@@1@17@@danf@4-5-2012 224001540280@unknown@formal@none@1@S@Many of these features are simple patterns; the most interesting (for this post) is my "MMR-like" feature.@@@@1@17@@danf@4-5-2012 224001540290@unknown@formal@none@1@S@MMR (Maximal Marginal Relevance) is a now standard technique in summarization that aims to allow your sentence extractor to extract sentences that aren't wholly redundant.@@@@1@25@@danf@4-5-2012 224001540300@unknown@formal@none@1@S@The way it works is as follows.@@@@1@7@@danf@4-5-2012 224001540310@unknown@formal@none@1@S@We score each sentence.@@@@1@4@@danf@4-5-2012 224001540320@unknown@formal@none@1@S@We pick as our first sentence the sentence with the highest score.@@@@1@12@@danf@4-5-2012 224001540330@unknown@formal@none@1@@We the rescore each sentence to a weighted linear combination of the original score and minus the similarity between the proposed second sentence and its similarity to the first.@@@@0@29@@danf@4-5-2012 224001540340@unknown@formal@none@1@S@Essentially, we want to punish redundancy, weighted by some parameter a.@@@@1@11@@danf@4-5-2012 224001540350@unknown@formal@none@1@S@This parameter is something that I tune in max-Rouge training.@@@@1@10@@danf@4-5-2012 224001540360@unknown@formal@none@1@S@What I found was that at the end of the day, the value of a that is found by the system is always negative, which means that instead of disfavoring redundancy, we're actually favoring it.@@@@1@35@@danf@4-5-2012 224001540370@unknown@formal@none@1@S@I always took this as a notion that human summaries really aren't that diverse.@@@@1@14@@danf@4-5-2012 224001540380@unknown@formal@none@1@@The take-home message is that if you can opportunistically pick one good sentence to go in your summary, the remaining sentences you choose should be as similar to that one was possible.@@@@0@32@@danf@4-5-2012 224001540390@unknown@formal@none@1@S@It's sort of an exploitation (not exploration) issue.@@@@1@8@@danf@4-5-2012 224001540391@unknown@formal@none@1@S@The problem is that I don't think this is true.@@@@1@10@@danf@4-5-2012 224001540400@unknown@formal@none@1@S@I think it's an artifact, and probably a pretty bad one, of the "new" version of Rouge with multiple references.@@@@1@20@@danf@4-5-2012 224001540410@unknown@formal@none@1@S@In particular, suppose I opportunistically choose one good sentence.@@@@1@9@@danf@4-5-2012 224001540420@unknown@formal@none@1@S@It will match a bunch of ngrams in, say, reference 1.@@@@1@11@@danf@4-5-2012 224001540430@unknown@formal@none@1@S@Now, suppose as my second sentence I choose something that is actually diverse.@@@@1@13@@danf@4-5-2012 224001540440@unknown@formal@none@1@S@Sure, maybe it matches something diverse in one of the references.@@@@1@11@@danf@4-5-2012 224001540450@unknown@formal@none@1@S@But maybe not.@@@@1@3@@danf@4-5-2012 224001540460@unknown@formal@none@1@S@Suppose instead that I pick (roughly) the same sentence that I chose for sentence 1.@@@@1@15@@danf@4-5-2012 224001540470@unknown@formal@none@1@S@It won't re-match against ngrams from reference 1, but if it's really an important sentence, it will match the equivalent sentence in reference 2.@@@@1@24@@danf@4-5-2012 224001540480@unknown@formal@none@1@S@And so on.@@@@1@3@@danf@4-5-2012 224001540490@unknown@formal@none@1@S@So this is all nice, but does it happen?@@@@1@9@@danf@4-5-2012 224001540500@unknown@formal@none@1@S@It seems so.@@@@1@3@@danf@4-5-2012 224001540510@unknown@formal@none@1@S@Below, I've taken all of the systems from DUC 2006 and plotted (on X) their human-graded Non-Redundancy scores (higher means less redundant) against (on Y) their Rouge-2 scores.@@@@1@28@@danf@4-5-2012 224001540520@unknown@formal@none@1@S@ Here, we clearly see (though there aren't even many data points) that high non-redundacy means low Rouge-2.@@@@1@18@@danf@4-5-2012 224001540530@unknown@formal@none@1@S@Below is Rouge-SU4, which is another version of the metric: @@@@1@11@@danf@4-5-2012 224001540531@unknown@formal@none@1@S@Again, we see the same trend.@@@@1@6@@danf@4-5-2012 224001540540@unknown@formal@none@1@S@If you want high Rouge scores, you had better be redundant.@@@@1@11@@danf@4-5-2012 224001540550@unknown@formal@none@1@S@The point here is not to gripe about the metric, but to point out something that people may not be aware of.@@@@1@22@@danf@4-5-2012 224001540560@unknown@formal@none@1@S@I certainly wasn't until I actually started looking at what my system was learning.@@@@1@14@@danf@4-5-2012 224001540570@unknown@formal@none@1@S@Perhaps this is something that deserves some attention.@@@@1@8@@danf@4-5-2012 224001720010@unknown@formal@none@1@S@Parallel Sampling@@@@1@2@@danf@4-5-2012 224001720020@unknown@formal@none@1@S@I've been thinking a lot recently about how to do MCMC on massively parallel architectures, for instance in a (massively) multi-core setup (either with or without shared memory).@@@@1@28@@danf@4-5-2012 224001720030@unknown@formal@none@1@S@There are several ways to approach this problem.@@@@1@8@@danf@4-5-2012 224001720040@unknown@formal@none@1@S@The first is the "brain dead" approach.@@@@1@7@@danf@4-5-2012 224001720050@unknown@formal@none@1@S@If you have N-many cores, just run N-many parallel (independent) samplers.@@@@1@11@@danf@4-5-2012 224001720060@unknown@formal@none@1@S@Done.@@@@1@1@@danf@4-5-2012 224001720070@unknown@formal@none@1@S@The problem here is that if N is really like (say 1000 or greater), then this is probably a complete waste of space/time.@@@@1@23@@danf@4-5-2012 224001720080@unknown@formal@none@1@S@The next approach works if you're doing something like (uncollapsed) Gibbs sampling.@@@@1@12@@danf@4-5-2012 224001720090@unknown@formal@none@1@S@Here, the Markov blankets usually separate in a fairly reasonable way.@@@@1@11@@danf@4-5-2012 224001720100@unknown@formal@none@1@S@So you can literally distribute the work precisely as specified by the Markov blankets.@@@@1@14@@danf@4-5-2012 224001720110@unknown@formal@none@1@@With a bit of engineering, you can probably do this is a pretty effective manner.@@@@0@15@@danf@4-5-2012 224001720120@unknown@formal@none@1@S@The problem, of course, is if you have strongly overlapping Markov blankets.@@@@1@12@@danf@4-5-2012 224001720130@unknown@formal@none@1@S@(I.e., if you don't have good separation in the network.)@@@@1@10@@danf@4-5-2012 224001720140@unknown@formal@none@1@S@This can happen either due to model structure, or due to collapsing certain variables.@@@@1@14@@danf@4-5-2012 224001720150@unknown@formal@none@1@S@In this case, this approach just doesn't work at all.@@@@1@10@@danf@4-5-2012 224001720160@unknown@formal@none@1@S@The third approach---and the only one that really seems plausible---would be to construct sampling schemes exclusively for a massively parallel architecture.@@@@1@21@@danf@4-5-2012 224001720170@unknown@formal@none@1@S@For instance, if you can divide your variables in some reasonable way, you can probably run semi-independent samplers that communicate on an as-needed basis.@@@@1@24@@danf@4-5-2012 224001720180@unknown@formal@none@1@S@The form of this communication might, for instance, look something like an MH-step, or perhaps something more complex.@@@@1@18@@danf@4-5-2012 224001720190@unknown@formal@none@1@S@At any rate, I've done a bit of a literature survey to find examples of systems that work like this, but have turned up surprisingly little.@@@@1@26@@danf@4-5-2012 224001720200@unknown@formal@none@1@S@I can't imagine that there's that little work on this problem, though.@@@@1@12@@danf@4-5-2012 224002020010@unknown@formal@none@1@S@Destination: Singapore@@@@1@2@@danf@4-5-2012 224002020020@unknown@formal@none@1@S@Welcome to everyone to ACL!@@@@1@5@@danf@4-5-2012 224002020030@unknown@formal@none@1@S@It's pretty rare for me to end up conferencing in a country I've been before, largely because I try to avoid it.@@@@1@22@@danf@4-5-2012 224002020040@unknown@formal@none@1@S@When I was here last time, I stayed with Yee Whye, who was here at the time as a postdoc at NUS, and lived here previously in his youth.@@@@1@29@@danf@4-5-2012 224002020050@unknown@formal@none@1@S@As a result, he was an excellent "tour guide."@@@@1@9@@danf@4-5-2012 224002020060@unknown@formal@none@1@S@With his help, here's a list of mostly food related stuff that you should definitely try while here (see also the ACL blog):@@@@1@23@@danf@4-5-2012 224002020070@unknown@formal@none@1@S@
  • Pepper crab.@@@@1@2@@danf@4-5-2012 224002020080@unknown@formal@none@1@S@The easiest to find are the "No Signboard" restaurant chain.@@@@1@10@@danf@4-5-2012 224002020090@unknown@formal@none@1@S@Don't wear a nice shirt unless you plan on doing laundry.
    • @@@@1@11@@danf@4-5-2012 224002020100@unknown@formal@none@1@S@
  • Chicken rice.@@@@1@2@@danf@4-5-2012 224002020110@unknown@formal@none@1@S@This sounds lame.@@@@1@3@@danf@4-5-2012 224002020120@unknown@formal@none@1@S@Sure, chicken is kind of tasty.@@@@1@6@@danf@4-5-2012 224002020130@unknown@formal@none@1@S@Rice is kind of tasty.@@@@1@5@@danf@4-5-2012 224002020140@unknown@formal@none@1@S@But the key is that the rice is cooked in or with melted chicken fat.@@@@1@15@@danf@4-5-2012 224002020150@unknown@formal@none@1@S@It's probably the most amazingly simple and delicious dish I've ever had.@@@@1@12@@danf@4-5-2012 224002020151@unknown@formal@none@1@S@"Yet Kun" (or something like that) is along Purvis street.
    • @@@@1@10@@danf@4-5-2012 224002020160@unknown@formal@none@1@S@
  • Especially for dessert, there's Ah Chew, a Chinese place around Liang Seah street in the Bugis area (lots of other stuff there too).
    • @@@@1@23@@danf@4-5-2012 224002020170@unknown@formal@none@1@S@
  • Hotpot is another local specialty: there is very good spicy Szechuan hotpot around Liang Seah street.
    • @@@@1@16@@danf@4-5-2012 224002020180@unknown@formal@none@1@S@
  • For real Chinese tea, here.@@@@1@5@@danf@4-5-2012 224002020190@unknown@formal@none@1@S@(Funny aside: when I did this, they first asked "have you had tea before?"@@@@1@14@@danf@4-5-2012 224002020200@unknown@formal@none@1@S@Clearly the meaning is "have you had real Chinese tea prepared traditionally and tasted akin to a wine tasting?"@@@@1@19@@danf@4-5-2012 224002020210@unknown@formal@none@1@S@But I don't think I would ever ask someone "have you had wine before?"@@@@1@14@@danf@4-5-2012 224002020220@unknown@formal@none@1@S@But I also can't really think of a better way to ask this!)
    • @@@@1@13@@danf@4-5-2012 224002020230@unknown@formal@none@1@S@
  • Good late night snacks can be found at Prata stalls (eg., indian roti with curry).
    • @@@@1@15@@danf@4-5-2012 224002020240@unknown@formal@none@1@S@
  • The food court at Vivocity, despite being a food court, is very good.@@@@1@13@@danf@4-5-2012 224002020250@unknown@formal@none@1@S@You should have some hand-pressed sugar cane juice -- very sweet, but very tasty (goes well with some spicy hotpot).
    • @@@@1@20@@danf@4-5-2012 224002020260@unknown@formal@none@1@S@
  • Chinatown has good Chinese dessert (eg., bean stuff) and frog leg porridge.
    • @@@@1@12@@danf@4-5-2012 224002020270@unknown@formal@none@1@S@Okay, so this list is all food.@@@@1@7@@danf@4-5-2012 224002020280@unknown@formal@none@1@S@But frankly, what else are you going to do here?@@@@1@10@@danf@4-5-2012 224002020290@unknown@formal@none@1@S@Go to malls? :).@@@@1@4@@danf@4-5-2012 224002020310@unknown@formal@none@1@S@There's definitely nice architecture to be seen; I would recommend the Mosque off of Arab street; of course you have to go to the Esplanade (the durian-looking building); etc.@@@@1@29@@danf@4-5-2012 224002020320@unknown@formal@none@1@S@You can see a few photos from my last trip here.@@@@1@11@@danf@4-5-2012 224002020330@unknown@formal@none@1@S@Now, I realize that most of the above list is not particularly friendly to my happy cow friends.@@@@1@18@@danf@4-5-2012 224002020340@unknown@formal@none@1@S@Here's a list of restaurants that happy cow provides.@@@@1@9@@danf@4-5-2012 224002020350@unknown@formal@none@1@S@There are quite a few vegetarian options, probably partially because of the large Muslim population here.@@@@1@16@@danf@4-5-2012 224002020360@unknown@formal@none@1@S@There aren't as many vegan places, but certainly enough.@@@@1@9@@danf@4-5-2012 224002020370@unknown@formal@none@1@S@For the vegan minded, there is a good blog about being vegan in Singapore (first post is about a recent local talk by Campbell, the author of The China Study, which I recommend everyone at least reads).@@@@1@37@@danf@4-5-2012 224002020380@unknown@formal@none@1@S@I can't vouch for the quality of these places, but here's a short list drawn from Living Vegan:@@@@1@18@@danf@4-5-2012 224002020390@unknown@formal@none@1@S@
  • Mushroom hotpot at Ling Zhi
    • @@@@1@5@@danf@4-5-2012 224002020400@unknown@formal@none@1@S@
  • Fried fake meat udon noodles (though frankly I'm not a big fan of fake meat)
    • @@@@1@15@@danf@4-5-2012 224002020410@unknown@formal@none@1@S@
  • Green Pasture cafe; looks like you probably have to be a bit careful here in terms of what you order
    • @@@@1@20@@danf@4-5-2012 224002020420@unknown@formal@none@1@S@
  • Yes Natural; seems like it has a bunch of good options
    • @@@@1@11@@danf@4-5-2012 224002020430@unknown@formal@none@1@S@
  • Lotus Veg restaurant, seems to have a bunch of dim sum (see here, too)
    • @@@@1@15@@danf@4-5-2012 224002020440@unknown@formal@none@1@S@
  • If you must, there's pizza
    • @@@@1@5@@danf@4-5-2012 224002020450@unknown@formal@none@1@S@
  • And oh-my-gosh, there's actually veggie chicken rice, though it doesn't seem like it holds up to the same standards as real chicken rice (if it did, that would be impressive)
    • @@@@1@30@@danf@4-5-2012 224002020460@unknown@formal@none@1@S@Okay, you can find more for yourself if you go through their links :).@@@@1@14@@danf@4-5-2012 224002020470@unknown@formal@none@1@S@Enjoy your time here!@@@@1@4@@danf@4-5-2012 224002020480@unknown@formal@none@1@S@Quick update: Totally forgot about coffee.@@@@1@6@@danf@4-5-2012 224002020490@unknown@formal@none@1@S@If you need your espresso kick, Highlander coffee (49 Kampong Bahru Road) comes the most recommended, but is a bit of a hike from the conference area.@@@@1@27@@danf@4-5-2012 224002020500@unknown@formal@none@1@S@Of course, you could also try the local specialty: burnt crap with condensed milk (lots and lots of discussion especially on page two here).@@@@1@24@@danf@4-5-2012 224002440010@unknown@formal@none@1@S@Parsing with Transformations@@@@1@3@@danf@4-5-2012 224002440020@unknown@formal@none@1@S@I remember when I took my first "real" Syntax class, where by "real" I mean "Chomskyan."@@@@1@16@@danf@4-5-2012 224002440030@unknown@formal@none@1@S@It was at USC in Fall 2001, taught by Roumyana Pancheva.@@@@1@11@@danf@4-5-2012 224002440040@unknown@formal@none@1@S@It was hard as hell but I loved it.@@@@1@9@@danf@4-5-2012 224002440050@unknown@formal@none@1@S@However, as a computationally minded guy, I remember snickering to myself the whole time we were talking about movements that get you from deep structure to surface structure.@@@@1@28@@danf@4-5-2012 224002440060@unknown@formal@none@1@S@This stuff was all computationally ridiculous.@@@@1@6@@danf@4-5-2012 224002440070@unknown@formal@none@1@S@But why was it computationally ridiculous?@@@@1@6@@danf@4-5-2012 224002440080@unknown@formal@none@1@S@It was ridiculous because my mindset, and I think the mindset of most computational folks at the time, was that of n^3 CKY or Earley style parsing.@@@@1@27@@danf@4-5-2012 224002440090@unknown@formal@none@1@S@Namely exact parsing in a context free manner.@@@@1@8@@danf@4-5-2012 224002440100@unknown@formal@none@1@S@This whole idea of transformations would kill anything like that in a very bad way.@@@@1@15@@danf@4-5-2012 224002440110@unknown@formal@none@1@S@However, there's been a recent shift in attitudes.@@@@1@8@@danf@4-5-2012 224002440120@unknown@formal@none@1@S@Sure, people still do their n^3 parsing, but of course none of it is exact anyway (due to pruning).@@@@1@19@@danf@4-5-2012 224002440130@unknown@formal@none@1@S@But more than that, things like linear time parsing algorithms as popularized by people like Joakim Nivre and Kenji Sagae and Brian Roark and Joseph Turian, have proved very useful.@@@@1@30@@danf@4-5-2012 224002440140@unknown@formal@none@1@S@They work well, are incredibly efficient, and are easy to implement.@@@@1@11@@danf@4-5-2012 224002440150@unknown@formal@none@1@S@They're also a bit more psychologically plausible (as Eugene Charniak said recently "we don't know what people are doing, but they're definitely not doing CKY.").@@@@1@25@@danf@4-5-2012 224002440160@unknown@formal@none@1@S@So I'm led to wonder: could we actually do parsing in a transformational grammar using all the new stuff we know about (for instance) left-to-right parsing?@@@@1@26@@danf@4-5-2012 224002440161@unknown@formal@none@1@S@One thing that stands in our way, of course, is the stupid Penn Treebank, which was annotated only with very simple transformations (mostly noun phrase movements) and not really "deep" transformations as most Chomskyan linguists would recognize them.@@@@1@38@@danf@4-5-2012 224002440170@unknown@formal@none@1@S@But I think you could still do it.@@@@1@8@@danf@4-5-2012 224002440180@unknown@formal@none@1@S@It would end up as being partially unsupervised, but at least from a minimum description length perspective, I can either spend weights learning more special cases, or I can learn general transformational rules.@@@@1@33@@danf@4-5-2012 224002440190@unknown@formal@none@1@S@It would take some thought and effort to write it out and figure out how to actually optimize such a thing, but I bet it could be done in a semester.@@@@1@31@@danf@4-5-2012 224002440200@unknown@formal@none@1@S@So then the question is: aside from smaller models (potentially), is there any other reason to do it?@@@@1@18@@danf@4-5-2012 224002440210@unknown@formal@none@1@S@I can think of at least one: parsing non-declarative sentences.@@@@1@10@@danf@4-5-2012 224002440220@unknown@formal@none@1@S@Since almost all sentences in the Treebank are declarative, parsers do pretty crappy when tested on other things.@@@@1@18@@danf@4-5-2012 224002440230@unknown@formal@none@1@S@Slav Petrov had a paper at EMNLP 2010 on parsing questions.@@@@1@11@@danf@4-5-2012 224002440240@unknown@formal@none@1@S@Here is the abstract, which says pretty much everything:@@@@1@9@@danf@4-5-2012 224002440250@unknown@formal@none@1@S@... We show that dependency parsers have more difficulty parsing questions than constituency parsers.@@@@1@14@@danf@4-5-2012 224002440260@unknown@formal@none@1@S@In particular, deterministic shift-reduce dependency parsers ... drop to 60% labeled accuracy on a question test set.@@@@1@17@@danf@4-5-2012 224002440270@unknown@formal@none@1@S@We propose an uptraining procedure in which a deterministic parser is trained on the output of a more accurate, but slower, latent variable constituency parser (converted to dependencies).@@@@1@28@@danf@4-5-2012 224002440280@unknown@formal@none@1@S@Uptraining with 100K unlabeled questions achieves results comparable to having 2K labeled questions for training.@@@@1@15@@danf@4-5-2012 224002440290@unknown@formal@none@1@S@With 100K unlabeled and 2K labeled questions, uptraining is able to improve parsing accuracy to 84%, closing the gap between in-domain and out-of-domain performance.@@@@1@24@@danf@4-5-2012 224002440300@unknown@formal@none@1@S@Now, at least in principle, if you can parse declarative sentences, you should be able to parse questions.@@@@1@18@@danf@4-5-2012 224002440310@unknown@formal@none@1@S@At least if you know about some basic syntactic transformations in English.@@@@1@12@@danf@4-5-2012 224002440320@unknown@formal@none@1@S@(As an aside, the "uptraining" idea is almost exactly the same as the structure compilation idea that Percy, Dan and I had at ICML 2008, though Slav and colleagues apply it to a domain adaptation problem, while we just did simple semi-supervised learning.)@@@@1@43@@danf@4-5-2012 224002440330@unknown@formal@none@1@S@We have observed similar effects in the parsing of commands, such as "Put your head in a noose" where parsers -- even constituency ones -- really really want "Put" to be a noun!@@@@1@33@@danf@4-5-2012 224002440340@unknown@formal@none@1@S@Again, if you know simple transformations -- like subject dropping -- you should be able to parse commands if you can already parse declarations.@@@@1@24@@danf@4-5-2012 224002440350@unknown@formal@none@1@S@As with any generalization, the hope is that by realizing the generalization, you don't need to store so many specific cases.@@@@1@21@@danf@4-5-2012 224002440360@unknown@formal@none@1@S@So if you can learn that commands and questions are simple transformation on declarative sentences, and you can learn to parse declaratives, you should be able to handle the other case.@@@@1@31@@danf@4-5-2012 224002440370@unknown@formal@none@1@S@(Anticipating comments: yes, I know you could try to pre-transform your data, like they do in MT, but that's quite inelegant.@@@@1@21@@danf@4-5-2012 224002440380@unknown@formal@none@1@S@And yes, I know you could probably take the treebank and turn a lot of the sentences into commands or questions to create a new data set.@@@@1@27@@danf@4-5-2012 224002440390@unknown@formal@none@1@S@But that's kind of missing the point: I don't want to just handle commands or questions... I want to handle anything, even things that I might not have anticipated.)@@@@1@29@@danf@4-5-2012 224002470010@unknown@formal@none@1@S@Some thoughts on supplementary materials@@@@1@5@@danf@4-5-2012 224002470020@unknown@formal@none@1@S@Having the option of authors submitting supplementary materials is becoming popular in NLP/ML land.@@@@1@14@@danf@4-5-2012 224002470030@unknown@formal@none@1@S@NIPS was one of the first conferences I submit to that has allowed this; I think ACL allowed it this past year, at least for specific types of materials (code, data), and EMNLP is thinking of allowing it at some point in the near future.@@@@1@45@@danf@4-5-2012 224002470040@unknown@formal@none@1@S@Here is a snippet of the NIPS call for papers (see section 5) that describes the role of supplementary materials:@@@@1@20@@danf@4-5-2012 224002470050@unknown@formal@none@1@S@In addition to the submitted PDF paper, authors can additionally submit supplementary material for their paper...@@@@1@16@@danf@4-5-2012 224002470051@unknown@formal@none@1@S@Such extra material may include long technical proofs that do not fit into the paper, image, audio or video sample outputs from your algorithm, animations that describe your algorithm, details of experimental results, or even source code for running experiments.@@@@1@40@@danf@4-5-2012 224002470060@unknown@formal@none@1@S@Note that the reviewers and the program committee reserve the right to judge the paper solely on the basis of the 8 pages, 9 pages including citations, of the paper; looking at any extra material is up to the discretion of the reviewers and is not required.@@@@1@47@@danf@4-5-2012 224002470070@unknown@formal@none@1@S@(Emphasis mine.)@@@@1@2@@danf@4-5-2012 224002470080@unknown@formal@none@1@S@Now, before everyone goes misinterpreting what I'm about to say, let me make it clear that in general I like the idea of supplementary materials, given our current publishing model.@@@@1@30@@danf@4-5-2012 224002470081@unknown@formal@none@1@S@You can think of the emphasized part of the call as a form of reviewer protection.@@@@1@16@@danf@4-5-2012 224002470082@unknown@formal@none@1@S@It basically says: look, we know that reviewers are overloaded; if your paper isn't very interesting, the reviewers aren't required to read the supplement.@@@@1@24@@danf@4-5-2012 224002470090@unknown@formal@none@1@S@(As an aside, I feel the same thing happens with pages 2-8 given page 1 in a lot of cases :P.)@@@@1@21@@danf@4-5-2012 224002470091@unknown@formal@none@1@@I think it's good to have such a form a reviewer protection.@@@@0@12@@danf@4-5-2012 224002470100@unknown@formal@none@1@S@What I wonder is whether it also makes sense to add a form of author protection.@@@@1@16@@danf@4-5-2012 224002470101@unknown@formal@none@1@S@In other words, the current policy -- which seems only explicitly stated in the case of NIPS, but seems to be generally understood elsewhere, too -- is that reviewers are protected from overzealous authors.@@@@1@34@@danf@4-5-2012 224002470110@unknown@formal@none@1@S@I think we need to have additional clauses that protect authors from overzealous reviewers.@@@@1@14@@danf@4-5-2012 224002470120@unknown@formal@none@1@S@Why?@@@@1@1@@danf@4-5-2012 224002470130@unknown@formal@none@1@S@Already I get annoyed with reviewers who seem to think that extra experiments, discussion, proofs or whatever can somehow magically fit in an already crammed 8 page page.@@@@1@28@@danf@4-5-2012 224002470140@unknown@formal@none@1@S@A general suggestion to reviewers is that if you're suggesting things to add, you should also suggest things to cut.@@@@1@20@@danf@4-5-2012 224002470150@unknown@formal@none@1@S@This situation is exacerbated infinity-fold with the "option" of supplementary material.@@@@1@11@@danf@4-5-2012 224002470160@unknown@formal@none@1@S@There now is no length-limit reason why an author couldn't include everything under the sun.@@@@1@15@@danf@4-5-2012 224002470170@unknown@formal@none@1@S@And it's too easy for a reviewer just to say that XYZ should have been included because, well, it could just have gone in the supplementary material!@@@@1@27@@danf@4-5-2012 224002470180@unknown@formal@none@1@S@So what I'm proposing is that supplementary material clauses should have two forms of protection.@@@@1@15@@danf@4-5-2012 224002470190@unknown@formal@none@1@S@The first being the existing one, protecting reviewers from overzealous authors.@@@@1@11@@danf@4-5-2012 224002470200@unknown@formal@none@1@S@The second being the reverse, something like:@@@@1@7@@danf@4-5-2012 224002470210@unknown@formal@none@1@S@Authors are not obligated to include supplementary materials.@@@@1@8@@danf@4-5-2012 224002470220@unknown@formal@none@1@S@The paper should stand on its own, excluding any supplement.@@@@1@10@@danf@4-5-2012 224002470230@unknown@formal@none@1@S@Reviewers must take into account the strict 8 page limit when evaluating papers.@@@@1@13@@danf@4-5-2012 224002470240@unknown@formal@none@1@S@Or something like that: the wording isn't quite right.@@@@1@9@@danf@4-5-2012 224002470250@unknown@formal@none@1@S@But without this, I fear that supplementary materials will, in the limit, simply turn into an arms race.@@@@1@18@@danf@4-5-2012 225000250010@unknown@formal@none@1@S@Intro to CL Books ...@@@@1@5@@danf@4-5-2012 225000250020@unknown@formal@none@1@S@Bob Carpenter has blogged about a new Intro to IR book online here.@@@@1@13@@danf@4-5-2012 225000250030@unknown@formal@none@1@S@I'm looking forward to skimming it this weekend.@@@@1@8@@danf@4-5-2012 225000250031@unknown@formal@none@1@S@I would also recommend the Python based NLTK Toolkit.@@@@1@9@@danf@4-5-2012 225000250040@unknown@formal@none@1@S@Books and resources like these are generally geared towards people with existing programming background.@@@@1@14@@danf@4-5-2012 225000250050@unknown@formal@none@1@S@If a linguist with no programming skills is interested in learning some computational linguistics, Mike Hammond has written a couple of novice's intro books called Programming For Linguists.@@@@1@28@@danf@4-5-2012 225000250060@unknown@formal@none@1@S@A novice would be wise to start with Hammond's books, move to the NLTK tutorials, then move on to a more serious book like Manning et al.@@@@1@27@@danf@4-5-2012 225000250070@unknown@formal@none@1@S@And if you're at all curious about what a linguist might DO once she has worked through all that wonderful material, you might could go to my own most wonderful List of Companies That Hire Computational Linguists page here.@@@@1@39@@danf@4-5-2012 225000250080@unknown@formal@none@1@S@And if you're not challenged by any of that above, I dare you to read Bob's Type-Logical Semantics.@@@@1@18@@danf@4-5-2012 225000250090@unknown@formal@none@1@S@Go on, you think yer all smart and such.@@@@1@9@@danf@4-5-2012 225000250100@unknown@formal@none@1@S@I dare ya!@@@@1@3@@danf@4-5-2012 225000250110@unknown@formal@none@1@S@I read it the summer of 1999 with a semanticist, a logician, and a computer scientist and it made all of our heads hurt.@@@@1@24@@danf@4-5-2012 225000250120@unknown@formal@none@1@S@I still have Chapter 10 nightmares.@@@@1@6@@danf@4-5-2012 225000310010@unknown@formal@none@1@@I may have misstated Chatham’s belief’s below.@@@@0@7@@danf@4-5-2012 225000310011@unknown@formal@none@1@S@It’s not clear that he agrees with the claim I complained about.@@@@1@12@@danf@4-5-2012 225000310012@unknown@formal@none@1@S@But, his blog makes it clear that he believes this:@@@@1@10@@danf@4-5-2012 225000310020@unknown@formal@none@1@S@experience can be coded in a non-linguistic form, and that recoding into language is possible, at least over short delays@@@@1@20@@danf@4-5-2012 225000310030@unknown@formal@none@1@S@First, I didn’t realize it was at all controversial that experience can be coded in non-linguistic form.@@@@1@17@@danf@4-5-2012 225000310031@unknown@formal@none@1@S@Of course it can.@@@@1@4@@danf@4-5-2012 225000310032@unknown@formal@none@1@S@Does anyone doubt this?@@@@1@4@@danf@4-5-2012 225000310040@unknown@formal@none@1@S@Second, I have no clue what Chatham means by recoding into language.@@@@1@12@@danf@4-5-2012 225000310041@unknown@formal@none@1@S@Certainly thoughts and memories can be expressed by language, that should go without saying; but, Chatham seems to believe that at least some thoughts and memories are STORED in language form.@@@@1@31@@danf@4-5-2012 225000310042@unknown@formal@none@1@S@This sounds like the old “we think in language” argument.@@@@1@10@@danf@4-5-2012 225000310050@unknown@formal@none@1@S@I am not convinced that we think in language.@@@@1@9@@danf@4-5-2012 225000310051@unknown@formal@none@1@S@In fact, I seriously doubt we think in language.@@@@1@9@@danf@4-5-2012 225000310052@unknown@formal@none@1@S@I think language is always a post-thought process.@@@@1@8@@danf@4-5-2012 225000360010@unknown@formal@none@1@S@casting a wide net@@@@1@4@@danf@4-5-2012 225000360020@unknown@formal@none@1@S@For the first time, I used sitemeter to view the site hits for this blog.@@@@1@15@@danf@4-5-2012 225000360030@unknown@formal@none@1@S@I only set that up a week ago, so the hits are recent only, but the range of locales is surprising.@@@@1@21@@danf@4-5-2012 225000360040@unknown@formal@none@1@S@I'm bigger in India than I ever would have imagined.@@@@1@10@@danf@4-5-2012 225000360050@unknown@formal@none@1@S@I can guess by some of the locations which of my friends are the likely culprit (Eric, you are spending wayyyyyyy too much time reading this blog).@@@@1@27@@danf@4-5-2012 225000360060@unknown@formal@none@1@S@But some of these just have no explanation, other than Bloggers "Next Blog" button.@@@@1@14@@danf@4-5-2012 225000360070@unknown@formal@none@1@S@Here's a list of hit locations (for hits that lasted longer than 0.00 seconds, which were many, unfortunately).@@@@1@18@@danf@4-5-2012 225000360080@unknown@formal@none@1@S@Bombay, India Brooklyn, NY (USA) Cambridge, UK Haifa, Israel Honolulu, Hawaii (USA) Hyderabad, India Kinards, SC (USA) Krakw, Poland Leuven, Belgium Mamers, NC (USA) Melbourne, Australia New York, NY (USA) Pittsburgh, PA (USA) Saint Paul, MN (USA) San Diego, California (USA) Seattle, Washington (USA) Sunnyvale, CA (USA) Tokyo, Japan Tulsa, OK (USA) Woking, UK@@@@1@54@@danf@4-5-2012 225000370010@unknown@formal@none@1@S@Data, Datum, Dati, Datillium, Datsun@@@@1@5@@danf@4-5-2012 225000370020@unknown@formal@none@1@S@The folks over at Cognitive Daily have blogged about the number properties of the word "data", or rather, they have blogged about the nitpicky prescriptivist grammar complaints that inevitably attend comments on academic paper submissions.@@@@1@35@@danf@4-5-2012 225000370030@unknown@formal@none@1@@Predictably, the comments sections is filled with people ignoring the main point, and instead making the same prescriptivist claims about the alleged plurality of "data".@@@@0@25@@danf@4-5-2012 225000370040@unknown@formal@none@1@@My 2 cents (in their comments) was simply that the word "data" has evolved into a word like like "deer" or "moose" which can be either singular or plural.@@@@0@29@@danf@4-5-2012 225000380010@unknown@formal@none@1@S@Buffalo Buffalo Bayes@@@@1@3@@danf@4-5-2012 225000380020@unknown@formal@none@1@S@The (somewhat) famous Buffalo sentence below seems to say something about frequency and meaning, I’m just not sure what:@@@@1@19@@danf@4-5-2012 225000380030@unknown@formal@none@1@S@Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo@@@@1@8@@danf@4-5-2012 225000380040@unknown@formal@none@1@S@Since the conditional probability of “buffalo” in the context of “buffalo” is exactly 1 (hey, I ain’t no math genius and I didn’t actually walk through Bayes theorem for this so whaddoo I know; I’m just sayin’, it seems pretty obvious, even to The Lousy Linguist).@@@@1@46@@danf@4-5-2012 225000380050@unknown@formal@none@1@S@Also, there is no conditional probability of any item in the sentence that is not 1; so from where does structure emerge?@@@@1@22@@danf@4-5-2012 225000380051@unknown@formal@none@1@S@Perhaps the (obvious) point is that a sentence like this could not be used to learn language.@@@@1@17@@danf@4-5-2012 225000380052@unknown@formal@none@1@S@One needs to know the structures first in order to interpret.@@@@1@11@@danf@4-5-2012 225000380053@unknown@formal@none@1@S@Regardless of your pet theory of learning this sentence will crash your learner.@@@@1@13@@danf@4-5-2012 225000380060@unknown@formal@none@1@S@There are only two sets of cues that could help: orthographic and prosodic.@@@@1@13@@danf@4-5-2012 225000380070@unknown@formal@none@1@S@There are three capitalized words, so that indicates some differentiation, but not enough by itself.@@@@1@15@@danf@4-5-2012 225000380080@unknown@formal@none@1@S@A learner would have to have some suprasegmental prosodic information to help identify constituents.@@@@1@14@@danf@4-5-2012 225000380081@unknown@formal@none@1@S@But how much would be enough?@@@@1@6@@danf@4-5-2012 225000380090@unknown@formal@none@1@S@Imagine a corpus of English sentences along these polysemic lines (with prosodic phrases annotated).@@@@1@14@@danf@4-5-2012 225000380100@unknown@formal@none@1@S@Would prosodic phrase boundaries be enough for a learner to make some fair predictions about syntactic structure?@@@@1@17@@danf@4-5-2012 225000380110@unknown@formal@none@1@@UPDATE (Nov 16, 2009): It only now occurs to me, years later, that the the very first Buffalo has no preceding context "buffalo".@@@@0@23@@danf@4-5-2012 225000380120@unknown@formal@none@1@S@Better late than never??@@@@1@4@@danf@4-5-2012 225000710010@unknown@formal@none@1@S@YIKES! or The New Information Extraction@@@@1@6@@danf@4-5-2012 225000710020@unknown@formal@none@1@S@The term information extraction may be taking on a whole new meaning to the greater world than computational linguists would have it mean.@@@@1@23@@danf@4-5-2012 225000710021@unknown@formal@none@1@S@As someone working in the field of NLP, I think of information extraction as in line with the Wikipedia definition:@@@@1@20@@danf@4-5-2012 225000710030@unknown@formal@none@1@S@information extraction (IE) is a type of information retrieval whose goal is to automatically extract structured information, i.e. categorized and contextually and semantically well-defined data from a certain domain, from unstructured machine-readable documents.@@@@1@33@@danf@4-5-2012 225000710040@unknown@formal@none@1@S@But my colleague pointed out a whole new meaning to me a couple weeks ago, the day after an episode of the NBC sitcom My Name Is Earl aired (11/1/2007: Our Other Cops Is On!).@@@@1@35@@danf@4-5-2012 225000710041@unknown@formal@none@1@S@Thanks to the wonders of The Internets, I managed to find a reference to the sitcom’s usage at TV Fodder.com:@@@@1@20@@danf@4-5-2012 225000710050@unknown@formal@none@1@S@Information extraction in a post-9/11 world involves delving into the nether regions of suspected terrorists....@@@@1@15@@danf@4-5-2012 225000710060@unknown@formal@none@1@S@In other words: TORTURE!@@@@1@4@@danf@4-5-2012 225000710061@unknown@formal@none@1@S@The law of unintended consequences has brought the world of NLP and the so called War on Terror into sudden intersection (yes, there are "other" intersections... shhhhhhh, we don't talk about those).@@@@1@32@@danf@4-5-2012 225000710062@unknown@formal@none@1@S@Perhaps the term IE is obsolete in CL anyway.@@@@1@9@@danf@4-5-2012 225000710063@unknown@formal@none@1@S@Wikipedia described it as a subfield of IR.@@@@1@8@@danf@4-5-2012 225000710064@unknown@formal@none@1@S@Manning & Schütze’s new book on the topic is called Introduction to Information Retrieval , not Introduction to Information Extraction.@@@@1@20@@danf@4-5-2012 225000710065@unknown@formal@none@1@S@They define IR, on the link above, essentially as finding material that satisfies information needs (note: I'm not quoting directly because the book is not yet out).@@@@1@27@@danf@4-5-2012 225000710070@unknown@formal@none@1@S@Quibbling over names and labels of subfields is often entertaining, but it’s ultimately a fruitless endeavor.@@@@1@16@@danf@4-5-2012 225000710071@unknown@formal@none@1@S@I defer to Manning & Schütze on all things NLP.@@@@1@10@@danf@4-5-2012 225000710072@unknown@formal@none@1@S@Information Retrieval it is.@@@@1@4@@danf@4-5-2012 225000790010@unknown@formal@none@1@S@Andrew Sullivan, Please Take a Cog Sci Class!!!!@@@@1@8@@danf@4-5-2012 225000790020@unknown@formal@none@1@S@Even though he blogs at a mere undergrad level (I’m slightly higher, heehee) I basically respect Andrew Sullivan as a blogger.@@@@1@21@@danf@4-5-2012 225000790021@unknown@formal@none@1@S@He blogs about a diverse set of topics and has thoughtful and intelligent (even if controversial) comments and analysis.@@@@1@19@@danf@4-5-2012 225000790022@unknown@formal@none@1@S@And he’s prolific, to say the least (surely the advantage of being a professional blogger, rather than stealing the spare moment at work while your test suite runs its course).@@@@1@30@@danf@4-5-2012 225000790023@unknown@formal@none@1@S@That said, he can sometimes really come across as a snobbish little twit.@@@@1@13@@danf@4-5-2012 225000790024@unknown@formal@none@1@S@Like yesterday when he linked to an article about Shakespearean language which talks about a psycholinguistics study initiated by an English professor, Philip Davis; as is so often the case, the professor has wildly exaggerated the meaning of the study.@@@@1@40@@danf@4-5-2012 225000790025@unknown@formal@none@1@S@Please see Language Log’s post Distracted By The Brain for related discussion.@@@@1@12@@danf@4-5-2012 225000790026@unknown@formal@none@1@@Here’s crucial quote from that post:@@@@0@6@@danf@4-5-2012 225000790030@unknown@formal@none@1@S@The neuroscience information had a particularly striking effect on non-experts’ judgments of bad explanations, masking otherwise salient problems in these explanations.@@@@1@21@@danf@4-5-2012 225000790040@unknown@formal@none@1@S@My claim: the neuroscience study discussed in the Davis article distracts the reader from Davis’s essentially absurd interpretations, and Andrew Sullivan takes the bait, hook, line and sinker (and looks like a twit in the end).@@@@1@36@@danf@4-5-2012 225000790050@unknown@formal@none@1@S@The article does not go into the crucial details of the study, but it says that it involves EEG (electroencephalogram) and MEG (magnetoencephalograhy) and fMRI (Functional Magnetic Resonance Imaging) noting that only the EEG portion has been completed.@@@@1@38@@danf@4-5-2012 225000790051@unknown@formal@none@1@S@A pretty impressive array of tools for a single psycholinguistics study, I must say.@@@@1@14@@danf@4-5-2012 225000790052@unknown@formal@none@1@S@Most published articles in the field would involve one or maybe two of these, but all three for a single study?@@@@1@21@@danf@4-5-2012 225000790053@unknown@formal@none@1@S@Wow, impressive.@@@@1@2@@danf@4-5-2012 225000790060@unknown@formal@none@1@S@It’s not clear to me if this was a well designed study or not (my hunch is, no, it is a poorly designed study, but without the crucial details, I really don’t know).@@@@1@33@@danf@4-5-2012 225000790061@unknown@formal@none@1@S@However, it is undeniable that professor Davis has gone off the deep end of interpretation.@@@@1@15@@danf@4-5-2012 225000790062@unknown@formal@none@1@S@The study does not even involve Shakespearean English!!!@@@@1@8@@danf@4-5-2012 225000790063@unknown@formal@none@1@S@It involves Modern English!@@@@1@4@@danf@4-5-2012 225000790064@unknown@formal@none@1@S@Then Davis makes the following claims (false, all of them, regardless of the study):@@@@1@14@@danf@4-5-2012 225000790070@unknown@formal@none@1@S@["word class conversion"] is an economically compressed form of speech, as from an age when the language was at its most dynamically fluid and formatively mobile; an age in which a word could move quickly from one sense to another… (underlines added)@@@@1@42@@danf@4-5-2012 225000790080@unknown@formal@none@1@S@This is the classic English professor bullshit.@@@@1@7@@danf@4-5-2012 225000790090@unknown@formal@none@1@S@I don’t even know what “economically compressed” means (Davis gives no definition); it has no meaning to linguistics that I know of.@@@@1@22@@danf@4-5-2012 225000790091@unknown@formal@none@1@S@The quote also suggests Shakespeare’s English had some sort of magical linguistic qualities that today’s English does not possess.@@@@1@19@@danf@4-5-2012 225000790092@unknown@formal@none@1@S@FALSE!@@@@1@1@@danf@4-5-2012 225000790093@unknown@formal@none@1@S@Modern English allows tremendous productivity of constructions, neologisms, and ambiguity.@@@@1@10@@danf@4-5-2012 225000790094@unknown@formal@none@1@S@A nice introduction to ambiguity can be found here: Ambiguous Words by George A. Miller.@@@@1@15@@danf@4-5-2012 225000790095@unknown@formal@none@1@S@Davis ends with a flourish of artistic bullshit hypothesizing:@@@@1@9@@danf@4-5-2012 225000790100@unknown@formal@none@1@S@For my guess, more broadly, remains this: that Shakespeare's syntax, its shifts and movements, can lock into the existing pathways of the brain and actually move and change them—away from old and aging mental habits and easy long-established sequences.@@@@1@39@@danf@4-5-2012 225000790110@unknown@formal@none@1@S@Neuroplasticity is only just now being studied in depth and it’s far from well understood, but the study in question says NOTHING about plasticity!!!@@@@1@24@@danf@4-5-2012 225000790111@unknown@formal@none@1@S@There’s also no reason to believe that Shakespeare’s language does anything that other smart, well crafted language does not do.@@@@1@20@@danf@4-5-2012 225000790112@unknown@formal@none@1@S@And we’re a generation at least away from having the tools to study any of this.@@@@1@16@@danf@4-5-2012 225000790120@unknown@formal@none@1@S@I’m accustomed to simply letting these all too common chunks of silliness go without comment, but then Andrew had to slip in his unfortunate bit of snooty arrogance.@@@@1@28@@danf@4-5-2012 225000790121@unknown@formal@none@1@S@After pasting a chunk of the obvious linguistics bullshit on his site (then follow-up comments), he has to finish with "I knew all that already".@@@@1@25@@danf@4-5-2012 225000790130@unknown@formal@none@1@S@Exactly what did you know, Andrew?@@@@1@6@@danf@4-5-2012 225000790140@unknown@formal@none@1@S@Since all of the major claims Davis makes are obvious bullshit, what exactly do you claim to have had prior knowledge of?@@@@1@22@@danf@4-5-2012 225000790150@unknown@formal@none@1@S@What did Andrew know, and when did he know it?@@@@1@10@@danf@4-5-2012 225000790160@unknown@formal@none@1@S@Really, Andrew, did you never take so much as a single linguistics course during all your years at Harvard and Oxford?@@@@1@21@@danf@4-5-2012 225000790161@unknown@formal@none@1@S@The University at Maryland has excellent psycholinguists as does Georgetown.@@@@1@10@@danf@4-5-2012 225000790170@unknown@formal@none@1@S@Please, consider sitting in on a course, won’t you?@@@@1@9@@danf@4-5-2012 225000910010@unknown@formal@none@1@S@"filibuster"@@@@1@1@@danf@4-5-2012 225000910020@unknown@formal@none@1@S@Some words just make me giggle.@@@@1@6@@danf@4-5-2012 225001000010@unknown@formal@none@1@S@Linguistics Forum@@@@1@2@@danf@4-5-2012 225001000020@unknown@formal@none@1@S@I just discovered this forum called Linguistics Forum.@@@@1@8@@danf@4-5-2012 225001000030@unknown@formal@none@1@S@I only looked at a few of the posts and I was underwhelmed, but I've never been a forum-kind-of-guy, so my opinion should be of minimal interest to those of you who utilize these resources.@@@@1@35@@danf@4-5-2012 225001000040@unknown@formal@none@1@S@Just thought I'd pass it along.@@@@1@6@@danf@4-5-2012 225001040010@unknown@formal@none@1@S@Tigrigna Blog and Resources@@@@1@4@@danf@4-5-2012 225001040020@unknown@formal@none@1@S@I just discovered a blog by a student of the language Tigrinya Qeyḥ bāḥrī.@@@@1@14@@danf@4-5-2012 225001040021@unknown@formal@none@1@S@From his site,@@@@1@3@@danf@4-5-2012 225001040030@unknown@formal@none@1@S@Being from a small city in Canada (Halifax, Nova Scotia) I found it very difficult to learn the mother tongue of my parents, as there are few resources availible from which I can learn.@@@@1@34@@danf@4-5-2012 225001040040@unknown@formal@none@1@S@So, I decided to create a resource for myself, somewhere I could collect everything I know about the language and use it at my leisure.@@@@1@25@@danf@4-5-2012 225001040050@unknown@formal@none@1@S@I thought about using my limited knowledge on HTML to create a webpage, that way I could have easy access to my work wherever I go.@@@@1@26@@danf@4-5-2012 225001040060@unknown@formal@none@1@S@And from Ethnologue@@@@1@3@@danf@4-5-2012 225001040070@unknown@formal@none@1@S@Tigrigna -- A language of Ethiopia Population -- 3,224,875 in Ethiopia (1998 census).@@@@1@13@@danf@4-5-2012 225001040080@unknown@formal@none@1@S@2,819,755 monolinguals.@@@@1@2@@danf@4-5-2012 225001040090@unknown@formal@none@1@S@Region Tigray Province.@@@@1@3@@danf@4-5-2012 225001040100@unknown@formal@none@1@S@Also spoken in Eritrea, Germany, Israel.@@@@1@6@@danf@4-5-2012 225001040110@unknown@formal@none@1@S@Alternate names -- Tigrinya, Tigray Classification -- Afro-Asiatic, Semitic, South, Ethiopian, North Language use -- National language.@@@@1@17@@danf@4-5-2012 225001040120@unknown@formal@none@1@S@146,933 second-language speakers. Language development -- Literacy rate in first language: 1% to 10%.@@@@1@14@@danf@4-5-2012 225001040130@unknown@formal@none@1@S@Literacy rate in second language: 26.5%.@@@@1@6@@danf@4-5-2012 225001040140@unknown@formal@none@1@S@Ethiopic script.@@@@1@2@@danf@4-5-2012 225001040150@unknown@formal@none@1@S@Radio programs.@@@@1@2@@danf@4-5-2012 225001040160@unknown@formal@none@1@S@Grammar.@@@@1@1@@danf@4-5-2012 225001040170@unknown@formal@none@1@S@Bible: 1956. Comments -- Speakers are called 'Tigrai'.@@@@1@8@@danf@4-5-2012 225001410010@unknown@formal@none@1@S@May 13, 2008@@@@1@3@@danf@4-5-2012 225001410020@unknown@formal@none@1@S@(screen grab from Psycholinguistics Arena)@@@@1@5@@danf@4-5-2012 225001410030@unknown@formal@none@1@S@What the hell happened on May 13, 2008?@@@@1@8@@danf@4-5-2012 225001640010@unknown@formal@none@1@S@Pundit Plays Linguist. Fails.@@@@1@4@@danf@4-5-2012 225001640020@unknown@formal@none@1@S@(screen shot of a guest at McCain's BBQ.@@@@1@8@@danf@4-5-2012 225001640030@unknown@formal@none@1@S@Video here)@@@@1@2@@danf@4-5-2012 225001640040@unknown@formal@none@1@S@Political pundits almost pathologically believe they have greater influence than they really do.@@@@1@13@@danf@4-5-2012 225001640050@unknown@formal@none@1@S@Case in point, Talking Points Memo's editor and publisher and chief blogger Josh Marshal has been trying to promote the use of the phrase "ride the swing" as a metaphor for the case when "a reporter who has gotten way too cozy with a politician and has had their supposed objectivity affected" (original explanation here).@@@@1@55@@danf@4-5-2012 225001640060@unknown@formal@none@1@S@The phrase refers to a posh BBQ that McCain hosted at one of his Arizona ranches where journalists were treated to a very comfy social experience that bordered on bribery (click on "video here" below the pic).@@@@1@37@@danf@4-5-2012 225001640070@unknown@formal@none@1@S@As far as I can tell, Marshal is the primary pusher of the phrase and its most frequent user (a couple other examples here and here).@@@@1@26@@danf@4-5-2012 225001640080@unknown@formal@none@1@S@I suspect Marshal's linguistic campaign will fail.@@@@1@7@@danf@4-5-2012 225001640090@unknown@formal@none@1@S@Attempts by a single person to explicitly promote the use of a new metaphor are rarely successful.@@@@1@17@@danf@4-5-2012 225001640100@unknown@formal@none@1@S@This is not how language works.@@@@1@6@@danf@4-5-2012 225001640110@unknown@formal@none@1@S@Successful new coinages are generally adopted less self-consciously.@@@@1@8@@danf@4-5-2012 225001640120@unknown@formal@none@1@S@The process is not well understood, but examples like Marshal's are few and far between.@@@@1@15@@danf@4-5-2012 225001640130@unknown@formal@none@1@S@Additionally, there are already several good metaphors for related frames, such as "drank the cool-aid" (which has equally obscure origins involving jungles and religious cults).@@@@1@25@@danf@4-5-2012 225001640140@unknown@formal@none@1@S@Not sure we need a new one just for journalists.@@@@1@10@@danf@4-5-2012 225001640150@unknown@formal@none@1@S@(HT to my colleague CC for bringing this to my attention.@@@@1@11@@danf@4-5-2012 225001640160@unknown@formal@none@1@S@At first, we had no clue what this metaphor referred to, and as such we literally couldn't understand what it was meant to evoke.@@@@1@24@@danf@4-5-2012 225001640170@unknown@formal@none@1@S@CC did some blogger detective work and discovered its origin).@@@@1@10@@danf@4-5-2012 225001810010@unknown@formal@none@1@S@Obama's Tango Conspiracy?@@@@1@3@@danf@4-5-2012 225001810020@unknown@formal@none@1@S@(screen shot from MSNBC's video)@@@@1@5@@danf@4-5-2012 225001810030@unknown@formal@none@1@S@Having nothing whatever to do with linguistics, nonetheless I feel compelled to report what seems like an entirely unreported snub by US President Barack Obama to the President of Argentina Cristina Kirchner.@@@@1@32@@danf@4-5-2012 225001810040@unknown@formal@none@1@S@Watch MSNBC's video of the second photo shoot and you'll see Obama walk across the entire group to go shake hands with Canada's PM Stephen Harper (who missed the original shoot), but he passed right in front of Kirchner who raised her hand out to shake Obama's, but he ignored her entirely (creating a somewhat awkward moment), shook Harper's hand, then refused to make eye contact with Kirchner afterwords.@@@@1@69@@danf@4-5-2012 225001810050@unknown@formal@none@1@S@I count that as two snubs.@@@@1@6@@danf@4-5-2012 225001810060@unknown@formal@none@1@S@Watch the video at Olbermann's "Countdown" site and at about 40 seconds in you'll see the moments I'm talking about.@@@@1@20@@danf@4-5-2012 225001810070@unknown@formal@none@1@S@MSNBC's footage seems to be the only one with a wide enough angle to show the snubs.@@@@1@17@@danf@4-5-2012 225001810080@unknown@formal@none@1@S@The relevant frootage is here: April 2, 2009; #5 "Obama meets the world press" (psssst, this has nothing to do with anything; just random rumor mongering...which is fun, ya know...)@@@@1@30@@danf@4-5-2012 225001840010@unknown@formal@none@1@S@Taco Verbs@@@@1@2@@danf@4-5-2012 225001840020@unknown@formal@none@1@S@(screen shot of this blog's Sitemeter data)@@@@1@7@@danf@4-5-2012 225001840030@unknown@formal@none@1@@A reader apparently was interested in "verbs that describes tacos."@@@@0@10@@danf@4-5-2012 225001840040@unknown@formal@none@1@S@Since the IP address shows the Indiana Department of Education, I got 20 bucks says this was done by a lunch lady writing out next week's menu.@@@@1@27@@danf@4-5-2012 225001840050@unknown@formal@none@1@S@As for the "linguistics aspect", well, verbs don't describe nouns (like "tacos"), adjectives do.@@@@1@14@@danf@4-5-2012 225001840060@unknown@formal@none@1@S@Verbs represent events.@@@@1@3@@danf@4-5-2012 225001840070@unknown@formal@none@1@S@Rather, adjectives describe nouns.@@@@1@4@@danf@4-5-2012 225001840080@unknown@formal@none@1@S@So, in the interest of serving my readers, exactly what kind of of adjectives describe tacos?@@@@1@16@@danf@4-5-2012 225001840090@unknown@formal@none@1@S@Let's go to the experts: Taco Bell:@@@@1@7@@danf@4-5-2012 225001840100@unknown@formal@none@1@S@
  • crunchy taco
    • @@@@1@2@@danf@4-5-2012 225001840110@unknown@formal@none@1@S@
  • soft taco
    • @@@@1@2@@danf@4-5-2012 225001840120@unknown@formal@none@1@S@
  • taco supreme(bonus points for postnominal adjective)
    • @@@@1@6@@danf@4-5-2012 225001840130@unknown@formal@none@1@S@
  • double decker taco.
    • @@@@1@3@@danf@4-5-2012 225002240010@unknown@formal@none@1@S@Lip Reading Response@@@@1@3@@danf@4-5-2012 225002240020@unknown@formal@none@1@S@This is a response to Liberman's Saturday morning goofiness here:@@@@1@10@@danf@4-5-2012 225002340010@unknown@formal@none@1@S@Regex Dictionary@@@@1@2@@danf@4-5-2012 225002340020@unknown@formal@none@1@S@Nice one!@@@@1@2@@danf@4-5-2012 225002340030@unknown@formal@none@1@S@A web-based dictionary you can search with regular expressions (HT MetaFilter).@@@@1@11@@danf@4-5-2012 225002340040@unknown@formal@none@1@S@from the site's introduction page: The Regex Dictionary is a searchable online dictionary, based on The American Heritage Dictionary of the English Language, 4th edition, that returns matches based on strings —defined here as a series of characters and metacharacters— rather than on whole words, while optionally grouping results by their part of speech.@@@@1@54@@danf@4-5-2012 225002340050@unknown@formal@none@1@S@For example, a search for "cat" will return any words that include the string "cat", optionally grouped according to gramatical category:@@@@1@21@@danf@4-5-2012 225002340051@unknown@formal@none@1@S@* Adjectives: catastrophic, delicate, eye-catching, etc.@@@@1@6@@danf@4-5-2012 225002340052@unknown@formal@none@1@S@* Adverbs: marcato, staccato, etc.@@@@1@5@@danf@4-5-2012 225002340053@unknown@formal@none@1@S@* Nouns: scat, category, vacation, etc.@@@@1@6@@danf@4-5-2012 225002340054@unknown@formal@none@1@S@* Verbs: cater, complicate, etc.@@@@1@5@@danf@4-5-2012 225002340060@unknown@formal@none@1@S@In other words, the Regex Dictionary searches for words based on how they are spelled; it can find:@@@@1@18@@danf@4-5-2012 225002340061@unknown@formal@none@1@S@* adjectives ending in ly (197; ex.: homely)@@@@1@8@@danf@4-5-2012 225002340062@unknown@formal@none@1@S@* words ending in the suffix ship (89)@@@@1@8@@danf@4-5-2012 225002340063@unknown@formal@none@1@S@o Adjectives (1, midship)@@@@1@4@@danf@4-5-2012 225002340064@unknown@formal@none@1@S@o Nouns (80; ex.: membership)@@@@1@5@@danf@4-5-2012 225002340065@unknown@formal@none@1@S@o Suffixes (1, -ship)@@@@1@4@@danf@4-5-2012 225002340066@unknown@formal@none@1@S@o Verbs (6; ex.: worship)@@@@1@5@@danf@4-5-2012 225002340067@unknown@formal@none@1@S@* words, not counting proper nouns, that have six consecutive consonants, including y (79; ex.: strychnine)@@@@1@16@@danf@4-5-2012 225002340068@unknown@formal@none@1@S@* words, not counting proper nouns, that have six consecutive consonants, not counting y (2; ex.: latchstring)@@@@1@17@@danf@4-5-2012 225002340069@unknown@formal@none@1@S@* words of 12 or more letters that consist entirely of alternate consonants and vowels (45; ex.: legitimatize)@@@@1@18@@danf@4-5-2012 225002700010@unknown@formal@none@1@S@All Of Them@@@@1@3@@danf@4-5-2012 225002730010@unknown@formal@none@1@S@Chinese Without Tone?@@@@1@3@@danf@4-5-2012 225002730020@unknown@formal@none@1@S@("please, wait a moment", image from braille.ch)@@@@1@7@@danf@4-5-2012 225002730030@unknown@formal@none@1@S@Vivian Aldridge has a nice website devoted to explaining braille systems for different languages (HT Boing Boing).@@@@1@17@@danf@4-5-2012 225002730040@unknown@formal@none@1@S@If I understand correctly, tone is rarely represented for Chinese braille (let's forgive for the moment that "Chinese" is the name of a language family, not a particular language):@@@@1@29@@danf@4-5-2012 225002730050@unknown@formal@none@1@S@In the few examples of Chinese braille that I have come across, the signs for the tones were not used except in the following cases:@@@@1@25@@danf@4-5-2012 225002730060@unknown@formal@none@1@S@
  • with the syllable yi, for which a good Chinese-German dictionary lists almost 50 different inkprint characters.@@@@1@16@@danf@4-5-2012 225002730070@unknown@formal@none@1@S@In this case the indication of the tone helps to limit the number of possible meanings.
    • @@@@1@16@@danf@4-5-2012 225002730080@unknown@formal@none@1@@
  • in words where a syllable with a suppressed vowel comes before syllable without a consonant, for example the word sh'yong (try out, test) in which the braille sign for the fourth tone is used instead of the apostrope.@@@@0@38@@danf@4-5-2012 225002730090@unknown@formal@none@1@S@In this case the tone sign seems to be used to separate the two syllables.
    • @@@@1@15@@danf@4-5-2012 225002730100@unknown@formal@none@1@S@Tone is a non-trivial feature of Chinese languages.@@@@1@8@@danf@4-5-2012 225002730110@unknown@formal@none@1@S@Omniglot has a nice page with the system displayed (fyi, it cites braille.ch as one of its sources).@@@@1@18@@danf@4-5-2012 225002730120@unknown@formal@none@1@S@The interesting point is that tone has the ability to be represented, but according to Vivian, it normally is not (however, she notes that she has only seen a few examples).@@@@1@31@@danf@4-5-2012 225002730130@unknown@formal@none@1@S@I spent two years in college struggling in Mandarin courses.@@@@1@10@@danf@4-5-2012 225002730140@unknown@formal@none@1@S@I would have liked to have dispensed with tone.@@@@1@9@@danf@4-5-2012 225003060010@unknown@formal@none@1@S@Correct All Grammar Errors And Plagiarism!@@@@1@6@@danf@4-5-2012 225003060020@unknown@formal@none@1@S@I was stupid enough to click through to Huffington Post's colossally stupid and fundamentally mistaken Worst Grammar Mistakes Ever post (I refuse to link to it).@@@@1@26@@danf@4-5-2012 225003060030@unknown@formal@none@1@S@Of course, the 11 items had virtually nothing to do with grammar (the vast majority were punctuation and spelling errors).@@@@1@20@@danf@4-5-2012 225003060040@unknown@formal@none@1@S@I must agree with Zwicky's pessimism regarding National Grammar Day: "It seems to me that the day is especially unlikely to provide a receptive audience for what linguists have to say."@@@@1@31@@danf@4-5-2012 225003060050@unknown@formal@none@1@S@But what prompted this post was the ad at the bottom for Grammarly, a free online "proofreader and grammar coach" which promised to Correct All Grammar Errors And Plagiarism.@@@@1@29@@danf@4-5-2012 225003060060@unknown@formal@none@1@S@A bold claim, indeed.@@@@1@4@@danf@4-5-2012 225003060070@unknown@formal@none@1@@I doubt a team of ten trained linguists could could felicitously make this claim.@@@@0@14@@danf@4-5-2012 225003060080@unknown@formal@none@1@S@But the boldness does not stop there (it never does on the innerwebz).@@@@1@13@@danf@4-5-2012 225003060090@unknown@formal@none@1@S@Click through to the online tool and wow, the bold claims just start stacking up like flap jacks at a Sunday fundraiser.@@@@1@22@@danf@4-5-2012 225003060100@unknown@formal@none@1@S@Just paste in your test and bam!@@@@1@7@@danf@4-5-2012 225003060110@unknown@formal@none@1@S@you get 150+ Grammar Checks@@@@1@5@@danf@4-5-2012 225003060111@unknown@formal@none@1@S@Get detailed error explanations.@@@@1@4@@danf@4-5-2012 225003060120@unknown@formal@none@1@S@Plagiarism Detection Find unoriginal text.@@@@1@5@@danf@4-5-2012 225003060130@unknown@formal@none@1@S@Text Enhancement Use better words.@@@@1@5@@danf@4-5-2012 225003060140@unknown@formal@none@1@S@Contextual Spell Check Spot misused words.@@@@1@6@@danf@4-5-2012 225003060150@unknown@formal@none@1@S@Dang!@@@@1@1@@danf@4-5-2012 225003060160@unknown@formal@none@1@S@Them fancy computers, they sure is smart.@@@@1@7@@danf@4-5-2012 225003060170@unknown@formal@none@1@@Just for funnin, I pasted the text of Zwicky's NGD again post into the window and ran the check.@@@@0@19@@danf@4-5-2012 225003060180@unknown@formal@none@1@S@Here's his report:@@@@1@3@@danf@4-5-2012 225003060190@unknown@formal@none@1@S@Not bad for a professor at one of the lesser linguistics departments.@@@@1@12@@danf@4-5-2012 225003060200@unknown@formal@none@1@S@(pssst, btw, did ya spot that odd little grey balloon at the top of the second screen shot?@@@@1@18@@danf@4-5-2012 225003060210@unknown@formal@none@1@S@Yeah, me too.@@@@1@3@@danf@4-5-2012 225003060220@unknown@formal@none@1@S@It says "click allow if presented with a browser security message."@@@@1@11@@danf@4-5-2012 225003060230@unknown@formal@none@1@S@Suspect, no doubt.@@@@1@3@@danf@4-5-2012 225003060240@unknown@formal@none@1@S@Nonetheless, I trusted Chrome to protect me and plowed ahead).@@@@1@10@@danf@4-5-2012 225003700010@unknown@formal@none@1@S@more on language death@@@@1@4@@danf@4-5-2012 225003700020@unknown@formal@none@1@S@Razib continues his thoughtful discussion of the interplay of linguistic diversity/homogeneity and socio-economic disparity/prosperity.@@@@1@14@@danf@4-5-2012 225003700030@unknown@formal@none@1@S@Money quote: If you have a casual knowledge of history or geography you know that languages are fault-lines around which intergroup conflict emerges.@@@@1@23@@danf@4-5-2012 225003700040@unknown@formal@none@1@S@But more concretely I’ll dig into the literature or do a statistical analysis.@@@@1@13@@danf@4-5-2012 225003700050@unknown@formal@none@1@S@I’ll have to correct for the fact that Africa and South Asia are among the most linguistically diverse regions in the world, and they kind of really suck on Human Development Indices.@@@@1@32@@danf@4-5-2012 225003700060@unknown@formal@none@1@S@And I do have to add that the arrow of causality here is complex; not only do I believe linguistic homogeneity fosters integration and economies of scale, but I believe political and economic development foster linguistic homogeneity.@@@@1@37@@danf@4-5-2012 225003700070@unknown@formal@none@1@S@So it might be what economists might term a “virtual circle.” (emphasis in original) I have a long history of discussing language death on this blog and my position can be summed up by this Q&A I had with myself:@@@@1@40@@danf@4-5-2012 225003700071@unknown@formal@none@1@S@Q: Is language death a separate phenomenon from language change?@@@@1@10@@danf@4-5-2012 225003700080@unknown@formal@none@1@S@A: In terms of linguistic effect, I suspect not@@@@1@9@@danf@4-5-2012 225003700081@unknown@formal@none@1@S@Q: Are there any favorable outcomes of language death?@@@@1@9@@danf@4-5-2012 225003700090@unknown@formal@none@1@S@A: I suspect, yes (Razib proposes one)@@@@1@7@@danf@4-5-2012 225003700091@unknown@formal@none@1@S@Q: How do current rates of language death compare with historical rates?@@@@1@12@@danf@4-5-2012 225003700100@unknown@formal@none@1@S@A: Nearly impossible to tell@@@@1@5@@danf@4-5-2012 225003700101@unknown@formal@none@1@S@Q: What is the role of linguists wrt language death?@@@@1@10@@danf@4-5-2012 225003700110@unknown@formal@none@1@S@A: One might ask: what is the role of mechanics wrt global warming?@@@@1@13@@danf@4-5-2012 225003790010@unknown@formal@none@1@S@verb valencies@@@@1@2@@danf@4-5-2012 225003790020@unknown@formal@none@1@S@A new online version of the 2004 book A Valency Dictionary of English has recently gone live.@@@@1@17@@danf@4-5-2012 225003790030@unknown@formal@none@1@S@I haven't had a chance to play with it, but it looks like it has some good data about verb patterns.@@@@1@21@@danf@4-5-2012 225003790040@unknown@formal@none@1@S@If you're into that kinda thing, I mean.@@@@1@8@@danf@4-5-2012 225003830010@unknown@formal@none@1@@son of of bitch, the weather is pickled@@@@0@8@@danf@4-5-2012 225003830020@unknown@formal@none@1@S@This is an awesome video of a Korean language professional teaching Korean speakers how to use swear words in English.@@@@1@20@@danf@4-5-2012 225003830030@unknown@formal@none@1@S@It's so good, it's pickled.@@@@1@5@@danf@4-5-2012 225003830040@unknown@formal@none@1@S@(HT kottke)@@@@1@2@@danf@4-5-2012 225003960010@unknown@formal@none@1@S@what a PhD looks like...@@@@1@5@@danf@4-5-2012 225003960020@unknown@formal@none@1@S@a pimple...@@@@1@2@@danf@4-5-2012 225003960030@unknown@formal@none@1@S@...and I remain happily ABD...@@@@1@5@@danf@4-5-2012 225003960031@unknown@formal@none@1@@See The illustrated guide to a Ph.D. for full set of images and discussion.@@@@0@14@@danf@4-5-2012 225004230010@unknown@formal@none@1@S@habits of mind@@@@1@3@@danf@4-5-2012 225004230020@unknown@formal@none@1@S@A high school math teacher helps us all understand critical thinking.@@@@1@11@@danf@4-5-2012 225004230030@unknown@formal@none@1@S@Personal fav: Looks at statements that are generally false to see when they are true.@@@@1@15@@danf@4-5-2012 225004230040@unknown@formal@none@1@S@(via kottke)@@@@1@2@@danf@4-5-2012 225004320010@unknown@formal@none@1@S@what wrong with citing a paper?@@@@1@6@@danf@4-5-2012 225004320020@unknown@formal@none@1@S@Hadas Shema, an Information Science PhD student, discusses some of the politics and problems with academic citations in The citation game.@@@@1@21@@danf@4-5-2012 225004320030@unknown@formal@none@1@S@She got her facts from Bornmann, L., & Daniel, H. (2008).@@@@1@11@@danf@4-5-2012 225004320040@unknown@formal@none@1@S@What do citation counts measure?@@@@1@5@@danf@4-5-2012 225004320050@unknown@formal@none@1@S@A review of studies on citing behavior Journal of Documentation, 64 (1).@@@@1@12@@danf@4-5-2012 225004320060@unknown@formal@none@1@S@This one jumped out at me: Journal-dependent factors: Getting published in a high-factor journal doesn't necessarily mean your paper is the best thing since sliced bread, but it means more people are likely to think so.@@@@1@36@@danf@4-5-2012 225004320070@unknown@formal@none@1@S@Also, the first paper in a journal usually gets cited more often (I wonder if that's still relevant, given how wide-spread electronic access is these days) (emphasis added).@@@@1@28@@danf@4-5-2012 225004320080@unknown@formal@none@1@S@Lot's of crap has been published in major journals.@@@@1@9@@danf@4-5-2012 225004320090@unknown@formal@none@1@@And the corollary deserves to be mentioned: lots of good article ar published in minor journals and fail to get the respect or notice they deserve.@@@@0@26@@danf@4-5-2012 225004590010@unknown@formal@none@1@S@a brief history of stanford linguistics dissertations@@@@1@7@@danf@4-5-2012 225004590020@unknown@formal@none@1@S@The above image comes from the Stanford Dissertation Browser and is centered on Linguistics.@@@@1@14@@danf@4-5-2012 225004590030@unknown@formal@none@1@S@This tool performs some kind of textual analysis of Stanford dissertations: every dissertation is taken as a weighted mixture of a unigram language model associated with every Stanford department.@@@@1@29@@danf@4-5-2012 225004590040@unknown@formal@none@1@S@This lets us infer, that, say, dissertation X is 60% computer science, 20% physics, and so on...@@@@1@17@@danf@4-5-2012 225004590041@unknown@formal@none@1@S@Essentially, the visualization shows word overlap between departments measured by letting the dissertations in one department borrow words from another department..@@@@1@21@@danf@4-5-2012 225004590050@unknown@formal@none@1@S@Thus, the image above suggests that Linguistics borrows more words from Computer Science, Education, and Psychology than it does from other disciplines.@@@@1@22@@danf@4-5-2012 225004590060@unknown@formal@none@1@S@What was most interesting was using the Back button to creating a moving picture of dissertation language over the last 15 years.@@@@1@22@@danf@4-5-2012 225004590070@unknown@formal@none@1@S@you'll see a lot of bouncing back and forth.@@@@1@9@@danf@4-5-2012 225004590080@unknown@formal@none@1@S@Stats makes a couple jumps here and there.@@@@1@8@@danf@4-5-2012 225004590090@unknown@formal@none@1@S@HT Razib Khan@@@@1@3@@danf@4-5-2012 225004780010@unknown@formal@none@1@S@Choose Your Own Career in Linguistics@@@@1@6@@danf@4-5-2012 225004780020@unknown@formal@none@1@S@Trey Jones* at Speculative Grammarian invites y'all to play his cute, and yet somewhat depressing, game: Choose Your Own Career in Linguistics.@@@@1@22@@danf@4-5-2012 225004780030@unknown@formal@none@1@S@As a service to our young and impressionable readers who are considering pursuing a career in linguistics, Speculative Grammarian is pleased to provide the following Gedankenexperiment to help you understand the possibilities and consequences of doing so.@@@@1@37@@danf@4-5-2012 225004780040@unknown@formal@none@1@S@For our old and bitter readers who are too far along in their careers to have any real hope of changing the eventual outcome, we provide the following as a cruel reminder of what might have been.@@@@1@37@@danf@4-5-2012 225004780041@unknown@formal@none@1@S@Let the adventure begin...@@@@1@4@@danf@4-5-2012 225004780042@unknown@formal@none@1@S@*hehe, he used to work at Cycorp, hehe...@@@@1@8@@danf@4-5-2012 225005030010@unknown@formal@none@1@S@do you despise eReaders and have tons of extra cash@@@@1@10@@danf@4-5-2012 225005030020@unknown@formal@none@1@S@...then this is for you: The Penguin Classics Complete Library is a massive box set consisting of nearly every Penguin Classics book ever published and is available on Amazon for only (only!)@@@@1@32@@danf@4-5-2012 225005030030@unknown@formal@none@1@S@$13,413.30.@@@@1@1@@danf@4-5-2012 225005030040@unknown@formal@none@1@S@A rundown:@@@@1@2@@danf@4-5-2012 225005030050@unknown@formal@none@1@S@
  • 1,082 titles
    • @@@@1@2@@danf@4-5-2012 225005030060@unknown@formal@none@1@S@
  • laid end to end they would hit the 52-mile mark
    • @@@@1@10@@danf@4-5-2012 225005030070@unknown@formal@none@1@S@
  • 700 pounds in weight
    • @@@@1@4@@danf@4-5-2012 225005030080@unknown@formal@none@1@S@
  • 828 feet if you stacked them
    • @@@@1@7@@danf@4-5-2012 225005030090@unknown@formal@none@1@S@
  • They arrived in 25 boxes
    • @@@@1@5@@danf@4-5-2012 225005030100@unknown@formal@none@1@S@My only complaint would be that Penguin Classics tend to be crappy books physically.@@@@1@14@@danf@4-5-2012 225005030110@unknown@formal@none@1@S@HT Kottke.@@@@1@2@@danf@4-5-2012