data wrangler
1010 stories
·
31 followers

Julie Andrews Teaches a Group of Young Puppets About the Performing Arts in ‘Julie’s Greenroom’

1 Comment

Julie Andrews and celebrity guests like Alec Baldwin, Ellie Kemper, and Josh Groban teach a group of young puppets about the performing arts in the Netflix series Julie’s Greenroom produced in collaboration with the Jim Henson Company.

The first season of series premieres on the streaming service March 17, 2017.

Read the whole story
digdoug
1 day ago
reply
Julie Andrews and Jim Henson Company? CAN I GIVE NETFLIX MORE MONEY?
Louisville, KY
Share this story
Delete

AI learns to write its own code by stealing from other programs

3 Comments
Comments
self code
Set a machine to program a machine

iunewind/Alamy Stock Photo

By Matt Reynolds

OUT of the way, human, I’ve got this covered. A machine learning system has gained the ability to write its own code.

Created by researchers at Microsoft and the University of Cambridge, the system, called DeepCoder, solved basic challenges of the kind set by programming competitions. This kind of approach could make it much easier for people to build simple programs without knowing how to write code.

“All of a sudden people could be so much more productive,” says Armando Solar-Lezama at the Massachusetts Institute of Technology, who was not involved in the work. “They could build systems that it [would be] impossible to build before.”

Ultimately, the approach could allow non-coders to simply describe an idea for a program and let the system build it, says Marc Brockschmidt, one of DeepCoder’s creators at Microsoft Research in Cambridge, UK.

DeepCoder uses a technique called program synthesis: creating new programs by piecing together lines of code taken from existing software – just like a programmer might. Given a list of inputs and outputs for each code fragment, DeepCoder learned which pieces of code were needed to achieve the desired result overall.

“It could allow non-coders to simply describe an idea for a program and let the system build it”

One advantage of letting an AI loose in this way is that it can search more thoroughly and widely than a human coder, so could piece together source code in a way humans may not have thought of. What’s more, DeepCoder uses machine learning to scour databases of source code and sort the fragments according to its view of their probable usefulness.

All this makes the system much faster than its predecessors. DeepCoder created working programs in fractions of a second, whereas older systems take minutes to trial many different combinations of lines of code before piecing together something that can do the job. And because DeepCoder learns which combinations of source code work and which ones don’t as it goes along, it improves every time it tries a new problem.

The technology could have many applications. In 2015, researchers at MIT created a program that automatically fixed software bugs by replacing faulty lines of code with working lines from other programs. Brockschmidt says that future versions could make it very easy to build routine programs that scrape information from websites, or automatically categorise Facebook photos, for example, without human coders having to lift a finger

“The potential for automation that this kind of technology offers could really signify an enormous [reduction] in the amount of effort it takes to develop code,” says Solar-Lezama.

But he doesn’t think these systems will put programmers out of a job. With program synthesis automating some of the most tedious parts of programming, he says, coders will be able to devote their time to more sophisticated work.

At the moment, DeepCoder is only capable of solving programming challenges that involve around five lines of code. But in the right coding language, a few lines are all that’s needed for fairly complicated programs.

“Generating a really big piece of code in one shot is hard, and potentially unrealistic,” says Solar-Lezama. “But really big pieces of code are built by putting together lots of little pieces of code.”

This article appeared in print under the headline “Computers are learning to code for themselves”

More on these topics:


Comments
Read the whole story
digdoug
1 day ago
reply
hope you enjoyed being vastly overpaid, business software developers.
Louisville, KY
skorgu
2 days ago
reply
Welp, it's basically doing my job already.
Share this story
Delete
1 public comment
dmierkin
2 days ago
reply
It finally happened :-)

Authors Can Learn a Lesson from the YouTubers Who Keep Imploding

1 Comment

Inspired by the recent self-destruction of Youtuber PewDiePie, Polygon published a long editorial yesterday which looked at why Youtube stars keep imploding.

According to the insider who wrote the piece, many of the fallen suffered from two common problems - problems which I think authors could learn from.

The first is that the pro Youtubers have to maintain a grueling daily schedule of shooting and uploading videos in order to not just keep their audience's attention but also to keep being punished by Youtube's algorithms.

For example: “subscriber burn,” which is a nefarious little side effect of not uploading a new video for a couple of weeks. The term was popularized by the Game Theory channel in 2014; your subscribers stop getting notified of your videos if they stop watching or you stop uploading. Going on vacation? Let’s hope you got a backlog, because you’ll see a big drop in views if you take a week or two off. And they might not come back.

...

Most uploaders begin to believe they have to flood the site with videos for a chance one goes viral or to reach subscribers who aren’t notified or to make up for losing them. And the numbers do go up when you start to do that, leaving many to believe it’s the only reliable way to keep relevant.

You need ad revenue if you want to make a living talking over video games, which means views and that means uploads. Or at the very least, you need brand deals which means you need clout, which means you need subscribers, which means views, which again means uploads. Most pros create at least one video a day, and it’s a punishing schedule. Some create as many as three videos a day.

Speaking from personal experience, I fully understand that need to get the content out today. It pushes us to go for "good enough" rather than the best work we can do.

And it's worse for pro Youtubers, who are under more pressure than news bloggers such as myself.  They have to produce content on a daily basis, and what's even worse is that they have to be photogenic, articulate, personable, and even worse - funny.

That might not sound like a difficult job; after all, it's what stand up comedians do, but Youtubers don't have the time to devote to perfecting each joke.

There’s an apparent double standard, right? Comedians tell AIDS jokes, Holocaust jokes, 9/11 jokes and much more. When a popular YouTuber does it, it’s suddenly being reported by the media (and, cough, other YouTubers). Didn’t George Carlin once say no topic is off limits?

Yeah. But like most comedians, he also spent a lot of his time writing those jokes, refining them, trying them in smaller clubs before his big venues, commiserating with his peers, etc. A “secret” of successful comedians is you don’t just spit out jokes that come to you. You develop bits, callbacks, sets, etc. There are legit reasons that Louis CK, Sarah Silverman, and Jim Jefferies get away with questionable jokes and JohnnySephiroth315 doesn’t.

So when PewDiePie tells a joke that goes wrong like using an Israeli company (Fiverr) to promote anti-semitism, it's not that he is an ass so much as there is a fundamental flaw in the process he uses to produce his videos - he has no easy way to beta-test his work:

You can actually imagine, if you like, PewDiePie doing a stand-up set and having comedian friends tell him at the bar that “man, you’ve been leaning on the Nazi stuff a bit lately.” Or an audience groaning at a smaller venue, which signals to him it’s time to do a rewrite. That’s why there are workshops, writing sessions and smaller venues and drinks with fellow comedians. You have to fail often when the stakes are low to learn how to get the big wins. It’s a process.

And like many Youtubers, PewDiePie has no opportunity to make small mistakes.

That is not to excuse his mistakes, however; my point is that PewDiePie's failure wasn't the specific videos but more general; it was the process he used to make the videos.

The general problem is that many Youtubers don't have a source of feedback which could help catch horrendously bad ideas before he uploads a video.

Which brings me to the second problem shared by many Youtubers.

Like bloggers, the pro Youtubers will often partner with agencies which sell ads and line up brand partnerships. Those agencies are called "Multi-Channel Networks", and according to the insider these agencies are strictly sales agents who do not fill the roles of business managers, image consults, producers, PR flacks, etc.

My MCN is typically pretty nice and in touch, but I’m not managed and if I decide to do an interview — or write this article — a PR person won’t notice or care. I’m completely on my own when it comes to thinking about how my audience views me, for better or worse. I don’t have a manager to call for advice, guidance or media training.

Surprisingly, this is also true of some of the biggest names in the business. I don’t want to make it sound like MCNs do nothing, they are valuable business partners that make it easier to pay the bills, but they definitely don’t curate your content. They don’t tell their big talent to “lay off the political posting,” or “dial it back on the hard stuff for a bit.” It’s all business, no grooming or advice.

I don’t think this is due to apathy or greed. I’m not sure they know how to handle these things either.

There are two lessons to be learned from this.

The first is that authors need to pay as much attention to their process as to their content; it's not just about publishing a book as it is about the steps authors go through.

How many editors are involved? What about beta readers?  Has anyone gotten a second opinion on the cover?

The second lesson to be learned is that authors need a safety net. The need to surround themselves with people who can catch mistakes before they blow up into huge public fiascos.

Luckily, most authors already have that safety net in the form of writing critique groups, beta readers, online forums like Absolute Write, and local writing clubs.

So authors have all sorts of ways to avoid blowing up their career - if they remember to take advantage of them.

image by iamchad

 


You just finished reading Authors Can Learn a Lesson from the YouTubers Who Keep Imploding which was published on The Digital Reader. If you liked what you read, how about joining the discussion?

Read the whole story
digdoug
2 days ago
reply
I can't seem to share the original polygon story... but this quote is so effing accurate about so much of life that the free distribution/ad based revenue stream has destroyed:

You have to fail often when the stakes are low to learn how to get the big wins. It’s a process.
Louisville, KY
Share this story
Delete

Manifestos and Monopolies – Stratechery by Ben Thompson

1 Comment

It is certainly possible that, as per recent speculation, Facebook CEO Mark Zuckerberg is preparing to run for President. It is also possible that Facebook is on the verge of failing “just like MySpace”. And while I’m here, it’s possible that UFOs exist. I doubt it, though.

The reality is that Facebook is one of the most powerful companies the tech industry — and arguably, the world — has ever seen. True, everything posted on Facebook is put there for free, either by individuals or professional content creators; and true, Facebook isn’t really irreplaceable when it comes to the generation of economic value; and it is also true that there are all kinds of alternatives when it comes to communication. However, to take these truths as evidence that Facebook is fragile requires a view of the world that is increasingly archaic.

Start with production: there certainly was a point in human history when economic power was derived through the control of resources and the production of scarce goods:

fullsizerender

However, for most products this has not been the case for well over a century; first the industrial revolution and then the advent of the assembly-line method of manufacturing resulted in an abundance of products. The new source of economic power became distribution: the ability to get those mass-produced products in front of customers who were inclined to buy them:

fullsizerender-2

Today the fundamental impact of the Internet is to make distribution itself a cheap commodity — or in the case of digital content, completely free. And that, by extension, is why I have long argued that the Internet Revolution is as momentous as the Industrial Revolution: it is transforming how and where economic value is generated, and thus where power resides:

img_0138

In this brave new world, power comes not from production, not from distribution, but from controlling consumption: all markets will be demand-driven; the extent to which they already are is a function of how digitized they have become.

This is why most Facebook-fail-fundamentalists so badly miss the point: that the company pays nothing for its content is not a weakness, it is a reflection of the fundamental reality that the supply of content (and increasingly goods) is infinite, and thus worthless; that the company is not essential to the distribution of products is not a measure of its economic importance, or lack thereof, but a reflection that distribution is no longer a differentiator. And last of all, the fact that communication is possible on other platforms is to ignore the fact that communication will always be easiest on Facebook, because they own the social graph. Combine that with the fact that controlling consumption is about controlling billions of individual consumers, all of whom will, all things being equal, choose the easy option, and you start to appreciate just how dominant Facebook is.

Given this reality, why would Zuckerberg want to be President? He is not only the CEO of Facebook, he is the dominant shareholder as well, answerable to no one. His power and ability to influence is greater than any President subject to political reality and check-and-balances, and besides, as Zuckerberg made clear last week, his concern is not a mere country but rather the entire world.

Facebook Unease

The argument that Facebook is more powerful than most realize is not a new one on Stratechery; in 2015 I wrote The Facebook Epoch that made similar points about just how underrated Facebook was, particularly in Silicon Valley. In my role as an analyst I can’t help but be impressed: I have probably written more positive pieces about Facebook than just about any other company, and frankly, still will.

And yet, if you were to take a military-type approach to analysis — evaluating Facebook based on capabilities, not intent — the company is, for the exact same reasons, rather terrifying. Last year in The Voters Decide I wrote:

Given their power over what users see Facebook could, if it chose, be the most potent political force in the world. Until, of course, said meddling was uncovered, at which point the service, having so significantly betrayed trust, would lose a substantial number of users and thus its lucrative and privileged place in advertising, leading to a plunge in market value. In short, there are no incentives for Facebook to explicitly favor any type of content beyond that which drives deeper engagement; all evidence suggests that is exactly what the service does.

The furor last May over Facebook’s alleged tampering with the Trending Topics box — and Facebook’s overwrought reaction to even the suggestion of explicit bias — seemed to confirm that Facebook’s incentives were such that the company would never become overtly political. To be sure, algorithms are written by humans, which means they will always have implicit bias, and the focus on engagement has its own harms, particularly the creation of filter bubbles and fake news, but I have longed viewed Facebook’s use for explicit political ends to be the greatest danger of all.

This is why I read Zuckerberg’s manifesto, Building a Global Community, with such alarm. Zuckerberg not only gives his perspective on how the world is changing — and, at least in passing, some small admission that Facebook’s focus on engagement may have driven things like filter bubbles and fake news — but for the first time explicitly commits Facebook to playing a central role in effecting that change in a manner that aligns with Zuckerberg’s personal views on the world. Zuckerberg writes:

This is a time when many of us around the world are reflecting on how we can have the most positive impact. I am reminded of my favorite saying about technology: “We always overestimate what we can do in two years, and we underestimate what we can do in ten years.” We may not have the power to create the world we want immediately, but we can all start working on the long term today. In times like these, the most important thing we at Facebook can do is develop the social infrastructure to give people the power to build a global community that works for all of us.

For the past decade, Facebook has focused on connecting friends and families. With that foundation, our next focus will be developing the social infrastructure for community — for supporting us, for keeping us safe, for informing us, for civic engagement, and for inclusion of all.

It all sounds so benign, and given Zuckerberg’s framing of the disintegration of institutions that held society together, helpful, even. And one can even argue that just as the industrial revolution shifted political power from localized fiefdoms and cities to centralized nation-states, the Internet revolution will, perhaps, require a shift in political power to global entities. That seems to be Zuckerberg’s position:

Our greatest opportunities are now global — like spreading prosperity and freedom, promoting peace and understanding, lifting people out of poverty, and accelerating science. Our greatest challenges also need global responses — like ending terrorism, fighting climate change, and preventing pandemics. Progress now requires humanity coming together not just as cities or nations, but also as a global community.

There’s just one problem: first, Zuckerberg may be wrong; it’s just as plausible to argue that the ultimate end-state of the Internet Revolution is a devolution of power to smaller more responsive self-selected entities. And, even if Zuckerberg is right, is there anyone who believes that a private company run by an unaccountable all-powerful person that tracks your every move for the purpose of selling advertising is the best possible form said global governance should take?

The Cost of Monopoly

My deep-rooted suspicion of Zuckerberg’s manifesto has nothing to do with Facebook or Zuckerberg; I suspect that we agree on more political goals than not. Rather, my discomfort arises from my strong belief that centralized power is both inefficient and dangerous: no one person, or company, can figure out optimal solutions for everyone on their own, and history is riddled with examples of central planners ostensibly acting with the best of intentions — at least in their own minds — resulting in the most horrific of consequences; those consequences sometimes take the form of overt costs, both economic and humanitarian, and sometimes those costs are foregone opportunities and innovations. Usually it’s both.

Facebook is already problematic for society when it comes to opportunity costs. While the Internet — specifically, the removal of distribution as a bottleneck — is the cause of journalism’s woes, it is Facebook that has gobbled up all of the profits in publishing. Twitter, a service I believe is both unique and essential, was squashed by Facebook; I suspect the company’s struggles for viability are at the root of the service’s inability to evolve or deal with abuse. Even Snapchat, led by the most visionary product person tech has seen in years, has serious questions about its long-term viability. Facebook is too dominant: its network effects are too strong, and its data on every user on the Internet too compelling to the advertisers other consumer-serving businesses need to be viable entities.

I don’t necessarily begrudge Facebook this dominance; as I alluded to above I myself have benefited from chronicling it. Zuckerberg identified a market opportunity, ruthlessly exploited it with superior execution, had the humility to buy when necessary and the audacity to copy well, and has deservedly profited in the face of continual skepticism. And further, as I noted, as long as Facebook was governed by the profit-maximization incentive, I was willing to tolerate the company’s unintended consequences: whatever steps would be necessary to undo the company’s dominance, particularly if initiated by governments, would have their own unintended consequences. And besides, as we saw with IBM and Windows, markets are far more effective than governments at tearing down the ecosystem-based monopolies they enable — in part because the pursuit of profit-maximizing strategies is a key ingredient of disruption.

That, though, is why for me this manifesto crosses the line: contra Spiderman, Facebook’s great power does not entail great responsibility; said power ought to entail the refusal to apply it, no matter how altruistic the aims, and barring that, it is on the rest of us to act in opposition.

Limiting Facebook

Of course it is one thing to point out the problems with Facebook’s dominance, but it’s quite another to come up with a strategy for dealing with it; too many of the solutions — including demands that Zuckerberg use Facebook for political ends — are less concerned with the abuse of power and more with securing said power for the “right” causes. And, from the opposite side, it’s not clear that a traditional antitrust is even possible for companies governed by Aggregation Theory, as I explained last year in Antitrust and Aggregation:

To briefly recap, Aggregation Theory is about how business works in a world with zero distribution costs and zero transaction costs; consumers are attracted to an aggregator through the delivery of a superior experience, which attracts modular suppliers, which improves the experience and thus attracts more consumers, and thus more suppliers in the aforementioned virtuous cycle…

The first key antitrust implication of Aggregation Theory is that, thanks to these virtuous cycles, the big get bigger; indeed, all things being equal the equilibrium state in a market covered by Aggregation Theory is monopoly: one aggregator that has captured all of the consumers and all of the suppliers.

This monopoly, though, is a lot different than the monopolies of yesteryear: aggregators aren’t limiting consumer choice by controlling supply (like oil) or distribution (like railroads) or infrastructure (like telephone wires); rather, consumers are self-selecting onto the Aggregator’s platform because it’s a better experience.

Facebook is a particularly thorny case, because the company has multiple lock-ins: on one hand, as per Aggregation Theory, Facebook has completely modularized and commoditized content suppliers desperate to reach Facebook’s massive user base; it’s a two-sided market in which suppliers are completely powerless. But so are users, thanks to Facebook’s network effects: the number one feature of any social network is whether or not your friends or family are using it, and everyone uses Facebook (even if they also use another social network as well).

To that end, Facebook should not be allowed to buy another network-based app; I would go further and make it prima facie anticompetitive for one social network to buy another. Network effects are just too powerful to allow them to be combined. For example, the current environment would look a lot different if Facebook didn’t own Instagram or WhatsApp (and, should Facebook ever lose an antitrust lawsuit, the remedy would almost certainly be spinning off Instagram and WhatsApp).

Secondly, all social networks should be required to enable social graph portability — the ability to export your lists of friends from one network to another. Again Instagram is the perfect example: the one-time photo-filtering app launched its network off the back of Twitter by enabling the wholesale import of your Twitter social graph. And, after it was acquired by Facebook, Instagram has only accelerated its growth by continually importing your Facebook network. Today all social networks have long since made this impossible, making it that much more difficult for competitors to arise.

Third, serious attention should be given to Facebook’s data collection on individuals. As a rule I don’t have any problem with advertising, or even data collection, but Facebook is so pervasive that it is all but impossible for individuals to opt-out in any meaningful way, which further solidifies Facebook’s growing dominance of digital advertising.


Anyone who has read Stratechery for any length of time knows I have great reservations about regulation; the benefits are easy to measure, but the opportunity costs are both invisible and often far greater. That, though, is why I am also concerned about Facebook’s dominance: there are significant opportunity costs to the social network’s dominance. Even then, my trepidation about any sort of intervention is vast, and that leads me back to Zuckerberg’s manifesto: it’s bad enough for Facebook to have so much power, but the very suggestion that Zuckerberg might utilize it for political ends raises the costs of inaction from not just opportunity costs to overt ones.

Moreover, my proposals are in line with Zuckerberg’s proclaimed goals: if the Facebook CEO truly wants to foster new kinds of communities, then he ought to unleash the force that can best build the tools those disparate communities might need. That, of course, is the market, and Facebook’s social graph is the key. That Zuckerberg believes Facebook can do it alone is evidence enough that for Zuckerberg, saving the world is at best a close second to saving Facebook; the last thing we need are unaccountable leaders who put their personal interests above those they purport to govern.

Read the whole story
digdoug
3 days ago
reply
"Facebook is already problematic for society when it comes to opportunity costs." A wee bit understated.
Louisville, KY
Share this story
Delete

If it looks like trumpery, smells like trumpery, and tastes like trumpery...

1 Comment
Here's an interesting case of Trumpery™  that I may use in the lecture tour and book.

Infowars, The Washington Times (WT) and other publications are reporting that “Nearly 2 million non-citizen Hispanics are illegally registered to vote.” Infowars adds that “a survey of Hispanics in the U.S. revealed as many as two million non-citizens are illegally registered to vote, reinforcing claims by President Donald Trump that millions of illegal votes were cast in the 2016 election.”*

What's the evidence for such a claim of 2 million of illegal voters? It's a bit complicated —and it's a stinky pile of bullshit— so please pay attention.

It all begins with this 2013 survey of 800 Hispanic adults conducted by Mclaughlin & Associates. The survey itself looks fine to me. I asked the author, John McLaughlin, and he provided detailed a explanations of the methodology, how the sample was randomly chosen, etc. My problem, then, is not the survey, but the far-fetched extrapolations that Infowars and WT made, and that unfortunately have gone viral.

On page 68 of the summary of results you will see that among the Hispanics in the sample who aren't citizens, 13% said that they are registered to vote:





The stories at Infowars and WT quote James D. Agresti, who leads a think-tank called Just Facts:

[Agresti] applied the 13 percent figure to 2013 U.S. Census numbers for non-citizen Hispanic adults. In 2013, the Census reported that 11.8 million non-citizen Hispanic adults lived here, which would amount to 1.5 million illegally registered Latinos.
Accounting for the margin of error based on the sample size of non-citizens, Mr. Agresti calculated that the number of illegally registered Hispanics could range from 1.0 million to 2.1 million.
“Contrary to the claims of many media outlets and so-called fact-checkers, this nationally representative scientific poll confirms that a sizable number of non-citizens in the U.S. are registered to vote,” Mr. Agresti said.

But Agresti's reasoning is faulty. He first took the 13%, then extrapolated to the entire 2013 population of Hispanics who aren't citizens (11.8 million,) and then used this tool to calculate the margin of error of the result.

This is wrong at multiple levels.

First, according to WT and InfoWars, 56% of people in the survey, 448 in total, were non-citizens. But that figure is incorrect. As the survey itself explains, they didn't ask all the 800 people about their citizenship. They only asked those people who were born outside of the United States. Here is the actual breakdown of approximate percentages and corresponding number of people, based on the documentation of the survey (see page 4):


So it's 263 non-citizens, not 448. Of those, 13% said they are registered to vote anyway. That is around 34 people out of a sample of 800.

This is worrying, but it's such tiny a number that it could be due to careless questioning, bad wording, etc. Those are crucial factors to ponder, as we don't know how the questions were posed, and whether all respondents understood them correctly. Moreover, the survey was designed with a specific purpose —asking Hispanics about politics— and it must be used just for that. If you want to analyze voter fraud, you ought to design a different survey. That's Survey Design 101 stuff.

Anyway, just for the sake of argument let's suppose that, regardless, we still decide to extrapolate from the 13% figure.

Even if the questions were very accurately worded, we'd need to consider the inevitable mathematical uncertainty of any data set before we apply its results to the entire population, not after. That's the reason why we report things such as confidence intervals and margins of error when discussing how a sample may correspond to the entire population it is drawn from. Statistician Heather Krause has written this excellent summary about how to use them, and about their many limitations. (Heather also sent me some corrections for this post. Thanks, Heather!)

I sent an e-mail to Agresti pointing out that his initial calculations were based on an incorrect number of non-citizen Hispanics, 448 instead of 263. He replied very graciously, acknowledged the mistake, and proposed this correction with a much larger margin of error:
For 2013, the year of the survey, the Census Bureau reports that 11,779,000 Hispanic non-citizens aged 18 and older resided in the United States. At a 13% registration rate, this is 1,531,270 Hispanic non-citizens registered to vote. Accounting for the sampling margin of error, there were about 264 non-citizens in this survey. In a population of 11.8 million, the margin of error for a sample of 264 is 6.0% with 95% confidence. Applied to the results of the survey, this is 824,530 to 2,238,010 Hispanic non-citizens registered to vote (with 95% confidence).

But this is still wrong.

The way to do an extrapolation like this —if you dare to do so; I wouldn't— is to calculate the margin of error of the original percentage, that tiny 13%, and then extrapolate to the entire population. And you shouldn't use the tool that Agresti chose, by the way. It's better to do it manually with pencil, paper, and a basic calculator.

Let me walk you through the process. First, the formula to calculate a confidence interval of a sample proportion is:



This looks much more complicated than it really is. First, that z value in there, called the “critical value,” is 1.96 when we want a confidence level of 95% —don't worry about where that comes from; if you want to learn more about it, read the middle chapters of The Truthful Art.

So, z = 1.96. Let's move on.

What about p? That is the proportion that those 34 non-citizen Hispanics who declared to be registered to vote represent over the entire sample of 800 Hispanics. Remember, those 34 are 13% of 263 non-citizens who were born outside of the U.S., but the percentage will be much smaller if we calculate it based on the entire sample: 34 out of 800 is 4%.

In case you forgot how to calculate percentages: (34/800)*100 = 4%.

In statistics we represent percentages as proportions of 1.0. Therefore, 4% becomes 0.04, and the 1-p in the formula becomes 0.96 (that's the remaining 96% of the sample of 800.)

Now that we know that z= 1.96 and p = 0.04, let's input them in the formula. Here is the result:


That 0.0135 should be read +/-13.5 after multiplying it by 100. That's the margin of error that surrounds the 13% figure.

Therefore, with a 95% confidence we can claim that, based on our survey, the percentage of non-citizen Hispanics in the entire population of 11.8 million who would claim to be registered to vote could be 13.5 percentage points smaller than 13%, or 13.5 percentage points larger.

Translated to English, this means that if we could conduct the same survey 100 times, in 95 of them we would obtain a percentage between 0% and 26.5% of non-citizen Hispanics who would claim to be registered to vote. Those are the percentages that we would need to extrapolate to the entire population: Between nobody and more than one quarter of the non-citizen Hispanic population could be registered to vote. This is such a wide confidence interval that you can simply discard it as absurd.

And that is without even considering that the 11.8 million figure from the Census also has its own margin of error, which I didn't even bother to check. Why wasting more energy?

All these figures are way, waaaaay too uncertain to say anything meaningful. Based on this data, we cannot claim that we have an illegal voter problem in the U.S., or that we don't. The data from the survey is useless for this purpose. And the survey wasn't designed to analyze voter fraud, anyway, as I wrote above.

Verdict: the InfoWars and Washington Times stories are pure and simple Trumpery™

*NOTE: I am not opposed to requiring a photo ID to vote in principle. There are arguments in favor and against it in the U.S: We have solid evidence that photo ID laws are used to restrict the vote of minorities (this book is a great starting point); but I also understand the concerns of those who want to keep elections 100% clean. I'm Spanish, and all Spaniards have a DNI (National Identification Document,) which you must show to vote. I can't see why this cannot happen in the U.S, too. There are some big and hairy “buts” in this comparison, though: Spain's DNI is extremely easy and inexpensive to get. And we are registered to vote by default.

Read the whole story
digdoug
4 days ago
reply
Whattttt... cherry picking statistics, and extrapolating upon a misunderstood data point? The horror. I really do love the name "Trumpery" though.
Louisville, KY
Share this story
Delete

Saga's Lying Cat Is the Official Mascot of 2017

io9
1 Comment and 7 Shares

Last year, as everything seemed to catch on fire all around us, it was clear that K.C. Green’s iconic “This is fine” dog was the animal totem of 2016. Now, we are already in desperate need of a new icon that can help us navigate the swift-moving, danger-filled reality of 2017, and I know what it is: Lying Cat.

Read more...

Read the whole story
adamcole
4 days ago
reply
Saw this somewhere on Twitter too.
Philadelphia, PA, USA
popular
4 days ago
reply
digdoug
5 days ago
reply
Louisville, KY
ridingsloth
5 days ago
reply
Share this story
Delete
Next Page of Stories