New Revelations from the Snowden Documents

Jake Appelbaum’s PhD thesis contains several new revelations from the classified NSA documents provided to journalists by Edward Snowden. Nothing major, but a few more tidbits.

Kind of amazing that that all happened ten years ago. At this point, those documents are more historical than anything else.

And it’s unclear who has those archives anymore. According to Appelbaum, The Intercept destroyed their copy.

I recently published an essay about my experiences ten years ago.

Posted on September 21, 2023 at 7:03 AM43 Comments

Comments

willmore September 21, 2023 7:19 AM

Archive? Are we talking the 2+GB ‘insurance’ file? Wasn’t there a torrent of that? I would think thousands of people have it.

Winter September 21, 2023 9:00 AM

Note that the PhD thesis of Jake Appelbaum can be found here at the Eindhoven University of Technology:
‘https://research.tue.nl/files/197416841/20220325_Appelbaum_hf.pdf

Dutch PhD theses are regular published books. This one is 333+ pages long.

It contains a lot of images that are labeled “Top Secret”. I am not sure whether Americans are even allowed to read it.

Clive Robinson September 21, 2023 9:44 AM

@ ALL,

Re : Jake Appelbaum’s PhD thesis

I was going to mention it from another article,

“Marvell disputes claim Cavium backdoored chips for Uncle Sam”

“Allegations date back a decade to leaked Snowden docs”

https://www.theregister.com/2023/09/19/marvell_disputes_claim_that_cavium/

But on finding Appelbaum mentioned and remembering the problems just mentioning his name has caused in the past…

But Jake did make an intetesting observation, that after his footnote blew up into a bit of a storm Marvell’s representatives “answered a question that nobody had asked”.

Which as the older and more cynical observers know is generally a sign of a “cover up” in progress…

Appelbaum went on to point out that the back door could have been one that had to be put in by way of being able to do business, and mentioned the NSA tampering with a NIST standard that had to be followed if equipment was to be sold.

As we know RSA went over and above this by taking a $10million payment from Spooksville for fixing it so that the backdoored algorithm in the standard,

“Was promoted as the prefered option”

Which is the sort of promotional behaviour that got Microsoft in serious hot water, and now Google and Meta are feeling the heat over, https://www.theregister.com/2023/09/20/google_on_trial_feds_challenge/

I considered using this Appelbaum dialog as a part fo rebutting the whiners over Chinese co-operation legislation, to show the US insists on having backdoors in US manufactured goods for their Mil / Intel and equivalent spying entities but without the honesty of legislation.

Which has come up yet again,

https://www.theregister.com/2023/09/20/huawei_china_claims/

At a most inopportune time for Washington and folks on the hill, who are on the wrong end of the “China Inside” and similar nonsense being used by them for a rather ill thought out sanctions based trade war (nobody ever accused Goppols of an over capacity in the common sense dept).

Denise Lamb September 21, 2023 10:37 AM

In 2014, the Internet Engineering Task Force (IETF) published RFC 7258, stating “Pervasive Monitoring Is an Attack”. As of last week, ietf.org shows only “Enable JavaScript and cookies to continue”. No explanation, no contact information, no privacy policy—just an order to reduce one’s browser privacy settings. Back then, they seemed open to things such as anonymous access, onion routing, etc. Have they officially reversed their position?

The computerweekly.com link shows exactly the same, as does rfc-editor.org. Is it even legal under the privacy laws of Europe and California to withhold all content, including the privacy policy and the contact details for a data controller, till one accedes to such demands?

Also, I don’t believe the IETF ever published JavaScript as a standard, which means their RFCs are no longer available via programs such as curl that implement only IETF standards. That seems kind of lame. Previously, one could’ve used a decades-old minimal TCP/IP stack and manually typed enough HTTP/1 to get, say, the HTTP/2 and TLS RFCs.

Clive Robinson September 21, 2023 10:50 AM

@ Bruce and the usual suspects,

Re : Follow your nose to find reward.

In the ComputerWeekly artical you will find a quote from Marvell’s “Stacey Keegan” which includes,

“Marvell also supports a wide variety of standard algorithms including several variants of AES, 3DES, SHA-2, SHA-3, RSA 2048, RSA 4096, RSA 8192, ECC p256/p384/p521, Kasumi, ZUC and SNOW 3G.”

Spot “SNOW 3G”[1] tucked in at the end…

It was,designed by well known backdoor placers in standards crypto used in “Over The Air”(OTA) interfaces ETSI-SAGE. So Appelbaum is very probably right and Marvell has been shipping backdoored algorithms courtesy of the French.

Anyway SNOW 3G is being phased out and replaced by SNOW V supposadly to meet the vastly increased data rates of 5G but not in hardware but software with a 128 or 256 bit key which lets be honest is realy very small for “stream ciphers” and 20Gbit data rates…

You can read more at,

https://www.ericsson.com/en/blog/2020/6/encryption-in-virtualized-5g-environments

[1] For those that do not know, SNOW 3G is another of those “stream ciphers” that use “Linear Feedback Shift Registers”(LFSRs) and a supposed nonlinear “Finite State Machine”(FSM) which is where all the security if any is to be found or not.

Backdooring the FSM such that the stream becomes quite linear thus breakable within 1024bits or so of stream generator output, is something that has been done before with OTA “privacy enhancing” crypto going provably back to A5/1 and A5/2,

https://www.ericsson.com/en/blog/2021/6/evolution-of-cryptographic-algorithms

And more recently the algorithms used in secure two way radios that were mentioned on this blog.

Denise Lamb September 21, 2023 11:24 AM

You can read more at, https://www.ericsson.com

Hmm… evidently, I can’t, not without enabling Javascript and cookies. And unlike the IETF, Ericsson—a Swedish company—is definitely subject to the E.U.’s ePrivacy Directive and GDPR. Anyone in Sweden feel like making a complaint to their local authority? You should be able to reproduce this with a fairly standard Tor Browser setup; just move the security slider to “safest”. They don’t seem to allow anonymous complaints, though.

Clive Robinson September 21, 2023 1:53 PM

@ Denise Lamb,

Re : Ericsson blog pages.

The two blog pages I give, both work without either cookies or javascript enabled in the UK (I never “paint myself as a target” by using Tor so I can not say about that).

As for the Ericsson web front page you give, it appears to need javascript[1] but not cookies to display. Because the marketing droids appear to think everyone wants wizzy wizzy windows flying on screen.

As for,

“Anyone in Sweden feel like making a complaint to their local authority?”

It would probably be better done in Germany under EU Disability legislation.

For some reason stealing PPI is seen as acceptable, but being unkind to disabled people is apparently discriminatory, thus grounds for being hung by your thumbs off of a street light… so gets corporate types twitchy.

Why this might be so I’ve no real idea, but hey if it works “to kill two birds with one stone” so much the better.

[1] I usually browse with both cookies and javascript turned off. Importantly because javascript takes a lot of battery life as well as being as slow as a hog with only two legs. The fact it also as a side effect stops adds being displayed and in a lot of cases also stops the dumbass “accept all cookies” nonsense overlay is just “cream and frosting on the cake” (make mine a heavy fruit cake matured with beer, rum and dark treacle 😉

Wannabe Techguy September 21, 2023 2:49 PM

“the NSA tampering with a NIST standard that had to be followed if equipment was to be sold.”

And yet I’ve been told, right here, that NIST could be trusted.

Denise Lamb September 21, 2023 5:46 PM

@ Clive,

I never “paint myself as a target” by using Tor so I can not say about that

To me, that seems a bit like deciding to always leave your curtains open, because otherwise people might start to wonder what you’re hiding.

I think we really need Tor or similar as a built-in browser feature. In a popular browser, not just Brave (which apparently became open-source without me noticing, though not yet packaged in Debian), so that using it wouldn’t be any more suspicious than opening a “private” window. That browser doesn’t seem likely to be Chrome, though perhaps Google would do it if regulators starting hassling them about privacy.

For a while, it seemed like the IETF were going to put anti-monitoring features in every new protocol (also see RFC 6973, via archive.org to avoid the ban). Within 5 years, though, they’d banned Tor users from viewing their site. And unbanned them after about month, so maybe that was a mistake, but now it’s happened again. It’s not a good sign if the IETF people don’t even notice, after a week, that their site is not anonymously viewable. And with the exact same message as several other sites, suggesting there’s some third party with enough access, even behind HTTPS, for the “pervasive monitoring” we were warned about.

Clive Robinson September 21, 2023 7:45 PM

@ Wannabe Techguy,

Sorry auto-mod has struck again so…

Part 1,

Re : NSA and NIST

“And yet I’ve been told, right here, that NIST could be trusted.”

It’s a long and realy quite sordid story of how the NSA abuses their charter because people in political positions have been fed what is effectively false information.

Clive Robinson September 21, 2023 7:50 PM

@ Wannabe Techguy,

Part 2,

The short version is NIST was legally required to consult the NSA, on maters involving Crypto and related subjects. As a result the NSA has atleast one sometimes more people working on individual committees.

Clive Robinson September 21, 2023 7:56 PM

@ Wannabe Techguy,

Part 3,

As we now know the NSA not only did not tell NIST the facts of what they were doing.

Clive Robinson September 21, 2023 8:01 PM

@ Wannabe Techguy,

Part 4,

Staff on the committees acting for the NSA are known to have behaved in improper ways to other other experts from the Open Community on the committee causing them in some cases to withdraw.

fib September 21, 2023 8:02 PM

Re Anonymization Software

It would be really interesting if the major distros shipped anonymization software by default. This would eliminate the very moment where the proverbial target is painted on one’s back, since it would relieve the user from the burden of having to positively act to download the application [intent]. It would offer an additional layer of deflection: I’m sorry, I’m just using this cool app that is available on my machine. Ain’t Linux great!?! 🙂

Clive Robinson September 21, 2023 8:03 PM

@ Wannabe Techguy,

Part 5,

Thus the NSA got their lies through into the standard. When non NSA experts pointed out the algorith concerned was flawed and could prove it NIST were in the humiliating position of having to withdraw the standard.

Clive Robinson September 21, 2023 8:05 PM

@ Wannabe Techguy,

Part 6,

Similar nonsense has happened in Europe with ETSI and their SAGE group and certain European crypto standards that go into all GSM phones and many digital two way radios.

Years ago I had involvment with standards and if you knew as I did you could watch the various nations “Sig Int Agencies” actively “tag teaming” each other.

As I’ve said before the usuall trick was to frame it in terms of “Health and Safety” such that anyone who tried to oppose their nonsense got treated to a variation of “think of the children” with the implication that you were a dispicable person not just unworthy but unfit to sit on a standards committee.

Clive Robinson September 21, 2023 8:07 PM

@ Wannabe Techguy,

Part 7,

There are other readers of this blog who have either seen this process in action or worse have been on the receiving end of it. They might speak up but the chances are they won’t untill they are sure their pension is safe…

When I was young I was taught that Civil Servants in the UK,

“Acted without fear or favour”

And,

“Spoke truth unto power”

Well if it was ever true it certainly is nolonger, and the same is probably true for all Western Nations these days. Especially where certain Civil Servants retire early and take up very lucrative corporate positions…

Back a long time ago long service civil servants who had behaved would get an honour like an MBE, OBE, Knighthood etc. These days they get sincure positions in corporates with payments atleast five times what would be their full 40year service pensions etc.

this guy September 22, 2023 12:55 AM

@willmore You’re thinking of Wikileaks.

Snowden seemed to not want the files shared directly with the public so he gave them to journalists. Greenwald was supposedly one of them. I wrote to Greenwald earlier this year and he claimed that the files would never be released except for maybe a few documents here and there. He claims Snowden didn’t want them released. So unless someone else dumps them online we’ll probably never see them at all.

Clive Robinson September 22, 2023 5:15 AM

@ Denise Lamb,

Re : Using Tor and guilt by association.

“To me, that seems a bit like deciding to always leave your curtains open, because otherwise people might start to wonder what you’re hiding.”

Which is something they are going to do if you leave the curtains closed all the time, or you only go out quietly late at night.

But also consider,

“Individual use of VPN’s and Tor usage especially has a bad reputation in the MSM”

So think how a prosecutor will see it and describe it to the press and jury, to ensure you become the next rung on their political ladder…

Tor use has a bad reputation because of it’s association with buying, accessing, or doing illegal things and it’s not going to be seen otherwise any time soon.

There’s that old saying in the UK and other Western nations about the less acceptable social activities, and worse, such as robbery with guns, gang violence, drug dealing and similar reported in the MSM of,

“We’re only five years behind America…”

Well you can flip that these days in the UK, US, and other WASP nations that have bought into the “War on xxx” thinking and the subsequent “The people are the enemy of the State” outlook, especially about “Societal Surveillance” activities by the State Guard Labour to,

“We’re only five years behind China or Russia”

Much of it pushed by dog whistle “Think of the children” argument.

Individual use of VPN’s did get some improvment to it’s image during “lockdown” with the vastly increased “Work From Home”(WFH) usage, where it was seen by employers as a necessity for corporate security. But with the idiotic push for “Back To Office”(BTO) working by “managment mantra” that is likely to wear off again.

Tor has such a bad reputation that as you’ve indicated increasing numbers of web sites are “black listing” connections from it. Similarly those Web-VPN’s that individuals have used to get around “regional content control” systems are getting black listed.

With “Intetnet Service Providers”(ISPs) especially in the US now getting into “we sell your On-Line activities to data brokers” game, any use of VPN’s or Tor will be “flagged for ever more”, and as “third party business records” is not subject to any sort of legal oversight protection like Guard Labour having to show “probable cause” before the judiciary.

Now you might consider this a “paranoid view point” some certainly will. I however consider it as “acting prudently” based on observing the way things are going.

But I never got into “Social Media” and was told I was effectively paranoid… Well peoples attitudes are changing as can be seen by the decreasing share price of “Social Media” organisations. Whilst you could argue Twitter/X is Hellon Rusk’s fault, a look at share prices from before his involvment were declining. Likewise what’s Meta now worth over the same period, about 25% of what it once was. But Alphabet/Google are very likely to have their share price take a bath due to all the legal issues they are running into.

Oh and I suspect anyone riding on the latest surveillance tool of AI LLM’s are going to get a near zero rating when it’s bubble bursts as it will in the not to distant future.

But an end thought for you,

Tor is realy not the best way to get anonymity or privacy On-Line. Especially with “forever records” being kept by those you probably have never heard of. Many were shocked about the Cambridge Analytica scandal but don’t realise about the likes of Palantir[1], and similar. For such organisations lockdown was a great boost to their privacy invading behaviours, as individuals got forced out from under the protection of corporate umbrellas.

[1] Palantir’s business plan is as I’ve indicated before, is basically to take the “Dealer of drugs to children” business model and apply it to the likes of Guard Labour. That is they take information and make it like a designer drug to sell as “inteligence” in an “arms length” fashion to get around laws designed to restrict Guard Labour and protect private citizens. That is Laws that were put in place to protect citizens from unwarranted intrusion into their lives and the modern equivalent of their homes, possessions and papers. As part of that business plan Palantir are also trying to make certain aspects of guard labour redundant, thus make what is left of guard labour organisations increasingly “dependent” on Palantir and their systems. Thus ensuring an increasingly substantial income from the public purse and a great deal of control over not just the guard labour but the citizens as well.

Clive Robinson September 22, 2023 5:35 AM

@ fib, ALL,

Re Anonymization Software

“It would be really interesting if the major distros shipped anonymization software by default.”

You left out the important bit of,

“and enabled it as the default connection method”

Otherwise the “intent” problem comes back again.

Many view the “HTTPS by default” campaign as a success. The reality is that it was not realy ever an “end user choice”, but a “content peovider” choice.

The ability to use HTTPS was long built in to nearly every commonly used way to access on-line resources but it was not being used for various reasons. It was only when the snowball was pushed hard enough that it started rolling down the mountain.

Anonymization software thus needs to be,

1, On and enabled on every PC by default.
2, Given a sufficient push.

Once over a tipping point it’s usage will become acceptable, but not otherwise.

Winter September 22, 2023 5:48 AM

@Clvie

So think how a prosecutor will see it and describe it to the press and jury, to ensure you become the next rung on their political ladder…

FUD, I would say.

In my country, the Netherlands, the number of directly connected users of Tor is ~70,000 daily on a population of 17 million.

Prosecutors nor police have the manpower to investigate them all, or even a sizeable fraction. Also, we do not have jury trials. In addition, privacy is “in the news” in the Netherlands and that includes our main news radio station doing items about Tor.[1]

I have yet to see a single story about a Dutch criminal trial where Tor was seen as an aggravating circumstance. Cryptophones are in the news all the time, but it is always the messages that are used in courts, not so much the sue of the phones.

So, I conclude: FUD

In other, less free countries like the USA where legal systems are generally dysfunctional, YMMV. But in democratic countries with a free press, no.

[1] ‘https://www.nporadio1.nl/nieuws/wetenschap-techniek/13579a50-2060-45eb-b348-493f4bb3efc6/privacy-is-belangrijkste-reden-voor-gebruik-tor-browser

fib September 22, 2023 8:57 AM

@Clive

You left out the important bit of,

“and enabled it as the default connection method”

Yes, that’s for sure. Thanks for appending it.

Denise Lamb September 22, 2023 10:31 AM

@ Clive Robinson,

[people getting suspicious] is something they are going to do if you leave the curtains closed all the time

I do leave my curtains closed all the time (about 90% of the way closed; enough to let in light, but not enough so people can see me). I don’t know whether people are suspicious of that, but nobody’s ever said anything. Maybe I’m on some police list of serial-curtain-closers, but I doubt it.

Aren’t you worried about “painting yourself as a target” by keeping Javascript disabled?

Tor is realy not the best way to get anonymity or privacy On-Line. Especially with “forever records” being kept by those you probably have never heard of.

Well, don’t leave us hanging. What’s a better way? I mean one that allows me to view existing web sites without them opting in—as, say, the perpetually-being-rewritten Freenet (and I think I2P) would require. Certainly I can’t use my real IP address for access, given the “forever records” being kept (by the NSA and friends, the content distribution networks, the advertisers, the analytics people, the image-hosters, the ISPs…). The intelligence agencies are staffed by fools if they’re not running at least a few major “private VPN” services that appear to compete with each other. The hosting companies have gotta be keeping logs to prevent spam and other abuse, and I’ve heard that some of them demand photo-ID scans (does anyone know a good virtual-private-server or e-mail host that doesn’t, and is located in a privacy-respecting country?).

lurker September 22, 2023 2:01 PM

@Denise Lamb

Applebaum mentions in his thesis “A well-known VPN provider, Mullvad …” who use a variant of wireGuard. They also do a “privacy” browser. It didn’t help that their website yesterday gave me three different error messages on three different browsers about a problem with their https implementation. They are up OK now, working for me without cookies or .js

FA September 22, 2023 2:31 PM

@Clive

I never “paint myself as a target” by using Tor so I can not say about that)

You paint yourself as a pretty bright target by what you write on this blog.

But I never got into “Social Media”

What is this blog other than a social medium ? Most of the content here (including yours) is not technical at all, it’s mostly opinion, rants, and selfies.

So I’d agree: FUD.

Clive Robinson September 22, 2023 5:25 PM

@ Winter,

“FUD, I would say.”

That’s an opinion about your jurisdiction made in your jurisdiction which is in “Continental Europe” and based on a quite different jurisprudence history that eminated from the English legal system that underlies the various WASP and Comenwealth nations jurisdiction behaviours.

What you need to realise is that in the US there are “appointed” Federal judges with life long tenure,

https://www.nbcnews.com/politics/politics-news/us-appeals-judge-96-suspended-rare-clash-fitness-rcna111314

Whilst in other places/States judges are elected, and where in some you need no qualifications of any form to hold the position of a judge or Sherif, as long as the “relevant” citizens are prepared to vote for you…

This “relevant” voting is as true for “appointed” Federal judges as it is for “elected” state judges as you can see in,

https://www.theatlantic.com/ideas/archive/2023/09/federal-judiciary-biden-court-appointments/675336/

And why the current Federal judges are almost alien to the US population denographics,

https://www.theatlantic.com/ideas/archive/2020/10/americas-judiciary-doesnt-look-like-america/616692/

So judges, sheriffs, and prosecutors who do not get elected for life, are often way more mindfull of “voter bias” and firting in with it than they realy should be. But even those who have tenure are still very mindfull of how they are seen politically as they can be selected for higher positions. Likewise those who aspire to such tenure, are keen to appear to be correct not just politicaly but by religion as well…

But also factor into the equation the US actually has litle to no real notion of “presumed inocence”. With local and main stream media running “trial by media” vigilantism / witchhunts on an almost continuous basis like a debased form of Roman Circus. So judged in the public eye on incompleate thus biased information and hearsay prior to an actual court trial… So even jurors are pre-biased by the media nonsense, thus fair trials are not exactly an expected outcome in the US judicial system. And this appears to be supported by the US conviction statistics.

JonKnowsNothing September 22, 2023 7:49 PM

@ Winter, @Clive, All

re: Elected v Appointed v Certified

USA: Pretty much the only places “certified” is really needed is in academia where PhDs Rule the Roost and medicine where Doctors of Medicine are rapidly losing any credibility for treating their patients (Medicine for Profit or Least Cost Medical Care).

Lawyers and “engineers” do need to pass a State Board test in their fields, however depending on what sort of work you are doing it may be moot.

For Judges on the Federal level, including the US Supreme Court there is ZERO requirement to know anything at all about the law. NONE. Of course the legal establishment sends in their appraisal and recommendations primarily to those who graduated from High End Universities, but it is not on the job requirements list.

States vary but most of them are elected, including State Supreme Courts. City Courts are elected also. It may help persuade voters that you have a degree in law but probably not a lot. Voters often pick from people who spent time as a prosecutor and generally will select a prosecutor over a defense attorney over a public defender. Folks rarely even know what cases they worked on but a 90% Murder Conviction Rate works far better than 90% Wrongful Arrests Reversed.

For President and such levels, you don’t need any qualifications at all. You don’t need to read, write, gone to school, graduated or any of that baggage. Acting works quite well as a lead-in to being The Leader: Sincerity is a good voting point.

The USA jettisoned a lot of the Old World Ideas when we started our 2nd Constitution. We didn’t particularly care for other people Lording It Over US. We do quite well on our own.

King Louis was the king of France before the revolution
And then he had his head cut off which spoiled his constitution

Haul Away for Rosie / Haul Away Joe
[ Roud 809 ; Ballad Index Doe004 ; Mudcat 71905 ; trad.]

Winter September 23, 2023 5:00 AM

@Clive

That’s an opinion about your jurisdiction made in your jurisdiction which is in “Continental Europe” and based on a quite different jurisprudence history that eminated from the English legal system that underlies the various WASP and Comenwealth nations jurisdiction behaviours.

If it is not FUD, document your warnings with actual cases where a random Tor user was singled out and attacked by LEO’s or prosecutors for using Tor.

I have given examples where millions of Tor users exist in Europe [1] without a single one dragged up in court for it.

Instead of scaring users away from protecting themselves online with baseless movie script threats, come up with real evidence that using Tor is dangerous. Until you do, I will call it FUD.

[1] Germany alone has 2.5M daily Tor users.

Winter September 23, 2023 5:21 AM

@JonknowsNothing

USA: Pretty much the only places “certified” is really needed is in academia where PhDs Rule the Roost and medicine where Doctors of Medicine are rapidly losing any credibility for treating their patients (Medicine for Profit or Least Cost Medical Care).

Teaching research without a PhD is like giving driving lessons without a driving license. Practicing medicine without passing the MD exams is positively homicidal.

States vary but most of them are elected, including State Supreme Courts.

Elected officials should not have to be certified. The whole point of an election is that the position is open to everyone. If you want certification, you must remove the election part.

fib September 23, 2023 11:40 AM

@Winter

Teaching research without a PhD is like giving driving lessons without a driving license.

I personally think this is equivalent to saying that knowledge needs gatekeepers. I would say that, in short, research is what humans do all the time, as a second nature. Carl Sagan beautifully described how hunter-gatherers are always making deductions. Those who draw the correct conclusions about reality through their method, whatever it may be, are the ones who survive to pass on their genes. My bottom line: An absolutely rational being does not need masters to reach knowledge.

And who mentors the original mentor?

Winter September 23, 2023 1:18 PM

@fib, JonKnowsNothing

I personally think this is equivalent to saying that knowledge needs gatekeepers. I would say that, in short, research is what humans do all the time, as a second nature.

No, but when you work in academia you teach students to do research. It a good practice to require from teachers that they have mastered what they teach.

Research is formulating a research question, collecting and analyzing data, statistics, writing and publishing papers about your research, writing proposals to get the means to do the research.

You can learn to do all that without completing a PhD. But can you teach it to students without ever having done the whole work yourself?

If you cannot finish your own PhD thesis, why should you be good at teaching students about finishing research projects?

Would you learn to be a chef from someone who never cooked for more than two people, or swimming from someone who cannot swim?

This was different in the past, as academia was much more informal and research was a lot cheaper to do. Nowadays, you have to finish your education before you are 30, and you cannot do research as a hobby in the evening hours.

Still, to teach research requires more than just being able to do research.

Winter September 23, 2023 1:32 PM

@fib

My bottom line: An absolutely rational being does not need masters to reach knowledge.

Knowledge about the world has to be learned from experience and passed on. The next generation adds their own experiences to the knowledge. Armchair science goes nowhere.

And who mentors the original mentor?

A teacher is not a mentor. A teacher collects knowledge and passes it on. So every generation has more knowledge to pass on.

JonKnowsNothing September 23, 2023 3:21 PM

@Winter, fib, all

re: Knowledge about the world has to be learned from experience and passed on.

Absolutely NOT. I’m LMAO at your points.

Modern academia teaches how to do research in a particular manner. How to use mathematics or statistical analysis to “prove a hypothesis”. All well and good, for a far as it goes.

The vast sums of humans do not get this sort of training and they still manage to invent things anyway. Not every invention is a great one and if it is really good, some neoliberal business consortium will squash it faster than an unnecessary machine juicer.

Things are not taught and passed on, SELECT things are taught and passed on. Only those items that have use either in academia or in RL terms are passed on.

  • eg: the many posts here about the global PanFamine; how to store foods, how to prepare dishes with minimal ingredients that taste good, and how to cook an egg or bacon on a ceramic tile square.

Knowledge is lost all the time. Sometimes it’s found again, sometimes we know what it is we found, but often we haven’t a clue.

  • eg: Mayan Civilization as a Utopian Peaceful Kingdom (changed a lot since 1980s when we learned to read some of the Mayan Glyphs. Many we still do not know.)
  • eg: The purpose of a flint tool (we don’t know yet, just good guesses)
  • eg: How to hew many ton stone blocks and fit them so tightly not even a blade of grass grows in the seams with stone tools. Try it with copper chisels as an upgrade.

Knowledge that is skill based is often lost. All the How To TV Shows about Victorian Era machinery. Very sophisticated mechanical systems running on water wheel or wind power. The Romans ground their wheat for bread using water wheel mills in series. The Dutch harnessed wind for their mills and it spread throughout Europe.

  • Can you make a water wheel? It doesn’t seem hard but it is a tricky business: Over the Top, Under the Wheel or Straight On. And then you have to figure out the gearing transfer.

The millwrights of old couldn’t read or write. The laborers that built the ancient monuments couldn’t read or write and as many were foreign immigrant labor they spoke different languages and had different measuring systems. Yet the temples went up and barring modern explosives many have stayed up.

  • Ever wonder how they dust the top of the ceiling and statues at the Vatican? Or raised the blocks for the great cathedrals before gasoline engines? Ever wonder how to hammer a helmet or dress a broadsword?

There are folks that do. Most do not have certificates and many will not pass along their knowledge because it is skill based. You can read how to knap a flint tool; doing it takes physical practice.

Your view is hampered by living within the modern vision of “human society” with technology on demand, with power to run the machines on demand, food on demand, housing on demand, entertainment on demand. Nothing wrong with this point of view, however, there’s a whole aspect you are missing.

  • Mayan basic numbers symbols use similar designs as other cultures. Amazing what a bar and a dot will calculate.

Winter September 23, 2023 5:15 PM

@JonKnowsNothing

The vast sums of humans do not get this sort of training and they still manage to invent things anyway.

There is a misconception here. Academia is first and foremost an educational “institution”. The Akademia was the name of the estate in classical Athens where Plato taught.

Universities are educational institutions and those working in research in university also have teaching duties. Hence the importance of having completed a PhD themselves before teaching others about it.

There is also a large body of history and philosophy of science and knowledge development outside of science, eg, Michel Foucault wrote a lot about this. But such knowledge is developed outside of academia. So there is no reason why a PhD would be of importance.

Winter September 24, 2023 3:45 AM

PS
@JonKnows

re: Knowledge about the world has to be learned from experience and passed on.

Absolutely NOT. I’m LMAO at your points.

If you know ways of knowledge accumulation that does not use experience, aka, empiricism, and “passing on”, please tell me. I would love to know about them.

n00bs September 27, 2023 2:41 PM

Why has the release of material from the Snowden files been in such a piecemeal fashion?

Why the secrecy by the actors who have had the files, if they are supposedly on the “good” side?

And are those files still available somewhere?

RobertT September 28, 2023 9:20 PM

Re Cavium
Semiconductor Chips being backdoored during the design phase, hmmm that seems unlikely, I mean, how would you even go about it?
Wouldn’t every junior engineer that even glanced at the code/database see the intentional vulnerabilities?
How would you ever get away with hiding backdoors on a chip?
What would you do with such a backdoor if it existed? I mean the code still has to run correctly, so what’s the point of the “backdooring” at the chip level?
Which chip functions would you try to backdoor?

Don’t ask me!….no I really mean, don’t ask me!!

Clive Robinson September 28, 2023 11:22 PM

@ RobertT,

“How would you ever get away with hiding backdoors on a chip?”

Depends on what you mean by “back-door”.

The allegation is apparebtly that “the standard was back-doored” like the Dual EC-DRBG that the NSA pushed into a NIST standard.

We know for a fact that ETSI had regularly issued under NDA “oh so secret” crypto algorithms for over thirty years under direct “French influance” and design. To be used in Cordless and Mobile Phones, Private Mobile Radio”(PMR) and other commercial communications systems. Especially anything that might be used “internationally” (look up A5/1 abd A5/2 as a starting point).

ETSI’s excuse, “it was a long time ago and it was to meet US requirments for weakend crypto” (yet they were still doing it after “Crypto Wars I” was over under the Clinton Administration).

So with regards,

1, Wouldn’t every junior engineer that even glanced at the code/database see the intentional vulnerabilities?

Maybe if they were very smart enough to spot an algorith trick as in the NSA Dual EC-DRBG, or the fact that there were two switched algorithms in “the standard”. But as they would be “working to standard” it would not be in most engineers employability interests to say anything.

2, How would you ever get away with hiding backdoors on a chip?

The same as with point 1.

3, What would you do with such a backdoor if it existed?

Well the French we know use the weak crypto to spy for “economic advantage”. Many years ago the head of their secret service admitted that such spying was less costly than R&D.

4, I mean the code still has to run correctly, so what’s the point of the “backdooring” at the chip level?

As it’s in the “standard” it is going to work correctly, so the weak crypto “for privacy” in the over the air interface can be either selectively or on mass be turned on to make spying on citizens and companies “Oh so easy”.

5, Which chip functions would you try to backdoor?

The same ones as the NSA and ETSI have already “back-doored”

A, The RNG used for keys, IVs, nonces and similar to take the actual entropy down from say 128bits equivalent to maybe 20bits equivalent. So enough entropy to keep those “not in the know” out, but in reality so little that it can be brut forced in nearly “real time”.

B, Make the crypto algorithm or the mode algorithm it’s used in only 40bits equivalent.

As I said it rather depends on what you consider a “back-door” to be… But we know there are effectively two types

1, Those that are official and in standards or protocols.
2, Those that are unofficial and used like “covert side channels” effectively “riding in the wake” of what appears to be secure.

An exanple of the second was the NSA conning NIST in the AES competition. Whilst the AES algorithm is CS theoretically, in a practical implementation it can be disastrously weak due to time based side channels. The faster / more efficient you make AES in hardware or software the worse the time based side channels have been.

Adam Young and Moti Yung, showed that there is so much redundancy in PubKey crypto that you can put in a covert channel that can be used to prime a brut force search on one of the primes. Thus reduce factoring to seconds or minutes rather than “more than the life of the universe”. Worse it’s a covert channel you can not show the existance for if you only have either the Public or Private keys or even both…

But as I’ve mentioned before to show just how usless “code reviews” are, I back-doored an encryption product that sent “serial data”… As you know ASCII is seven bit and the eight bit can be “none / even / odd” parity. Well that “parity bit” is redundant or can be easily made so, which means it becomes available as a “side channel”. Litter the source code with comments about how a “known bit” can be used to break a stream generator and pull a fast one[1] with regards “randomly filling” the parity bit in the cipher text so infact you send out the equivalent of a LFSR modulated with the key…

I’ve not tried it but I suspect that certain “hardware description languages”(HDLs) like verilog are going to have similar “tricks” that will end up in the “Register Transfer Logic”(RTL) that synthesizes the actual hardware. The question is will it pass under peoples noses…

But also in sufficiently complex designs “macros” will get “bought in” and will get treated like “black boxes” if someone can find a communications protocol that gets sent out into the world with sufficient redundancy in the ciphertext such that a side chanbel becomes available, then it’s game over.

On trick you could use is to apparently randomly send errors in the transmitted ciphertext that the distant end error correction removes. There are numerous ways you could modulate such a channel to turn it into a covert side channel that leaks the crypto key in some way…

And there are a couple of other tricks I can think of where you can find redundancy in a system you can use for a covert side channel to leak information.

[1] I used the trick of creating a buffer with malloc() to load the key into then use that to build the stream cipher state array. Then use free() to get rid of the buffer before exiting the key load&make function… Only free() does not actually get rid of the buffer and it’s contents, because if you immediately use malloc() again for the same size buffer mostly back then you would get given a pointer to the original buffer on the heap that still has the key in it. Back then very few programers knew about this issue, even though one of the most notorious bugs due to the difficulty of finding it was returning an invalidated pointer from a function after using free()… It mostly works without consequences as that part of the heap does not change untill the next malloc() which is why it did not get noticed a lot of the time.

RobertT September 29, 2023 2:46 AM

@CliveR
What’s the famous Benjamin Franklin quote
“The only way that three people can keep a secret is if two of them are dead”
there’s a lot of wisdom in these words and they also give us direction visavis, how to implement functionality that should remain secret.

If you wanted to backdoor something like a chip it seems to me that also you’d want to keep the fact that it was backdoored secret, even from other engineers with full access to the entire chip database. If nothing else it’s plausable deniability.

That would suggest that you’d try to keep the chip design database (verilog or whatever HDL) completely clean and focus instead on corrupting the system at the implementation stage.

This can be done at a number of ways, here’s a few that come to mind
– Test system interface (typically serial Jtag like structure)
– Analog to Digital interface (design databases are often messed up at the interface because of different tool sets / design flow being used)
– Actual onchip Analog and RF circuitry (bit of a black art for most)

If I was really trying to hide something I’d probably use a combination of the above. The serialized on-chip-test infrastructure is the perfect place to create hidden system states/configurations for the Analog circuits. These are links that shouldn’t be there but are if the correct bits are loaded into the test control registers.
Makes me wonder what Cavium did or didn’t do, and more importantly exactly who did or didn’t do it.

Clive Robinson September 29, 2023 8:22 AM

@ RobertT,

“If I was really trying to hide something I’d probably use a combination of the above.”

Those are what I would call “low level in the stack” attacks as they are right down on the “physical device” physics. And you would have to do it on a device by device basis as part of the design, or modify the design tools to somehow know where to build in the fault, which would not be that easy.

In effect they are “implementation attacks” and usually require someone “in the know” in the production chain for them to be able to happen.

Longer term readers might remember back rather more than a decade, when I pointed out that from the NSA perspective they needed “known plaintext” (this was in fact pointed out by Robert Morris the NSA chief scientist when he retired). Thus the Microsoft File Formats must have been a deep joy to them…

But I pointed out that from my perspective if I was a SigInt agency I would try to “finesse”

1, Standards
2, Protocols
3, Implementations.

I’ve pointed out that prior to the NSA existing the design of US mechanical field ciphers had strong keys through to very weak keys. As the “keys in use” were issued from a central source, they could ensure that only the strong keys were used by US forces. However any captured equipment that the enemy reused or copied would in all probability be used by those not sufficiently experienced to know which were strong and which were weak keys. This would enable US Cryptographas to easily read around 1/5 of the messages, and from this gain “probable plaintext” to more easily assist in attacking the use of even the strong keys using automated systems (similar to the Enigma bombs etc). The fact that many nations from the end of WWII into the 1980’s were still using those “war surplus” US, German and very similar mechanical crypto devices would have been a positive joy to the NSA, GCHQ, and other Five Eyes and related agencies (we also now know for certain that “Crypto AG” was influanced in this weak key idea all through it’s history).

But it’s not just crypto. This backdoor on a chip is what is known to have happened with a German manufacturer of line interface chips for the “Plain Old Telephone System”(POTS) that reproduced the old “capacitor across the hook switch trick”. Whereby low frequencies in the audio band saw an “open circuit” or atleast a very high impedence to a low impedence transmission line. Whilst frequencies in the LF through MF and into HF bands would see a “closed circuit” or atleast a very low impedence to the transmission line. Howrver arrange for a device behind that capacitor to be “impedence modulated” by the microphone and that HF current will get modulated. The use of the equivalent of a circulator in the transmission line will pull out that modulated current and give you the audio.

It was in effect one way to do your,

“Actual onchip Analog and RF circuitry”

To make it more of a “covert backdoor” it was not a “capacitor” equivalent but a “Twin T and gyrator”[1] equivalent to making a very narrow band pass at a frequency[2] outside of the frequency range used in the compliance standards for testing.

Moving to “protocols” as I’ve mentioned “error correction” at the transmission level makes an almost perfect covert backdoor. Because whilst it leakes at the transmission level to Eve etc, it never gets seen at the higher levels as error correction gets stripped from the data as it gets passed up the comms stack.

By it’s nature error correction is generally a highly specialised subject and few understand it or even want to understand it. The fact you can build it into a secure encryption system –see McEliece[3]– does not appear to have endeared it to people. Which brings us onto “standards” you can hide a backdoor in an error correction standard with little or no difficulty, you simply need to use the excuse of “whitening” / “randomization” to equalise the communications bandwidth or make it “Post Quantum Secure” and few if any will look further into it.

We know the NSA attack standards in various ways the most obvious being the Dual EC-DRBG and the humiliation it caused NIST.

Few others though have gone on to ask the all important question of,

“What other standards?”

As I’ve repeatedly pointed out it’s quite a lot… The excuse is generally a “Health and Safety Capability” which is why your GSM mobile phones truely are “bugs in your pocket”. But the one that made me laugh sardonically is a step or two up the computing stack, from “standards” to “legislation”. That is for “Health and Safety” the US decided all mobile phones would have to have GPS installed that could be accessed “in an emergency” by an operator…

I could go on but the point is most technology has two failings,

1, It’s dual use or can be presented as “for the common good”.
2, It all has “redundancy” that builds in side chanbels.

Few ever notice how easily “good becomes bad”, and even less ever ask “Just a moment…” because they don’t want to be seen as a nail that sticks up, thus get banged down or thrown out the door (a mistake I made early on in my career over “fingerprint scanners”).

[1] Whilst I’ve built such a Twin T circuit using more discrete components, I’ve not tried doing the equivalent with a Sallen–Key circuit but I suspect it will work.

[2] For those who are having trouble getting their heads around the idea imagine a transmission line terminated in a low pass filter a series switch and finally a variable impedence load such as a FET or Transistor to ground driven by the audio signal. This is a standard POTS telephone circuit to replace the old hook switch and carbon granual microphone. Now the “covert back door” is simply a series tuned RF circuit at a suitably high frequency and Q that goes from the transmission line to the variable impedence load, thus bypassing the lowpass audio filter and electronic hook switch. It’s the same trick as is used in those TAO so called “radar bugs” where the series tuned circuit is a dipole antenna with the low impedence mid point bridged by the variable impedence.

[3] Sadly Robert McEliece’s system from the 1970’s never gained favour with the open cryptographic community. Even now when it’s known to have “Quantum Computing”(QC) algorithm resistance it’s still more or less shuned for mostly spurious reasons,

https://en.m.wikipedia.org/wiki/McEliece_cryptosystem

Maureen Monroe September 29, 2023 12:38 PM

@ Clive Robinson,

Sadly Robert McEliece’s system from the 1970’s never gained favour with the open cryptographic community. Even now when it’s known to have “Quantum Computing”(QC) algorithm resistance it’s still more or less shuned for mostly spurious reasons

It gained enough favour to be entered into NIST’s post-quantum cryptography standardisation process. I had some trouble finding out why they ultimately rejected it—they apparently never mentioned it in any formal report. The answer is in an announcement: “Classic McEliece was a finalist [for Key-Establishment Mechanisms (KEMs)] but is not being standardized by NIST at this time. Although Classic McEliece is widely regarded as secure, NIST does not anticipate it being widely used due to its large public key size [around 1 megabyte]. NIST may choose to standardize Classic McEliece at the end of the fourth round.”

They also wrote “SIKE remains an attractive candidate for standardization because of its small key and ciphertext sizes”; readers might remember that SIKE failed catastrophically a month after that announcement was published.

There was no McEliece-based signature scheme submitted. Wikipedia says one could be created based on Niederreiter’s scheme, and while it doesn’t mention signature size, I believe it would simply be the size of whatever hash were used: probably 32 bytes. So this seems entirely practical for hundred-megabyte software projects (such as operating systems) or certificate authorities whose public keys would be pre-loaded. Not so good to use on every HTTPS site.

Clive Robinson September 29, 2023 4:36 PM

@ Maureen Monroe, ALL,

Re : McEliece Key size.

“Although Classic McEliece is widely regarded as secure, NIST does not anticipate it being widely used due to its large public key size [around 1 megabyte]. NIST may choose to standardize Classic McEliece at the end of the fourth round.”

They neglected to mention why the alleged key size is large, but when used in a sensibley designed communications system would not actually be of any real note.

You first have to realize in a conventional system encryption and error correction are done on entirely seperate levels and kept segregated. The McEliece system I worked with some years ago, actually combined encryption and error correction into one effective function.

So when you add back in all the extra bits for “error correction” you have to put in the communications channel for a conventional system it can be quite a shock. Thus what appears to be a large McEliece key size is not as big as other seperate encryption and error correction systems need.

Also with modern large media files, including modern “documents” the actual combined encryption and error correction bits added are actually not as significant as it first sounds.

Something tells me NIST’s advisors the NSA have already decided against McEliece, but have decided not to say. There are a couple of basic reasons to do this,

1, To limit the spread of encryption they can not break.

2, They have already broken it but won’t reveal the method. To stop it being reciprocated on US / Five Eye systems.

But at the end of the day it could be other reasons or none at all.

Maureen Monroe September 29, 2023 7:43 PM

The combination of encryption and error-correction doesn’t seem like something that would work well with existing workflows. We need something that works with TCP/IP (which gives us an already-error-corrected stream) and TLS (which does encryption and signing on top); and DNSSEC, DKIM, and various other protocols. A “sensibley designed communications system” that combines these layers is a non-starter. We can’t even get people to switch from IPv4 to IPv6, despite it existing for 27 years and us being out of IPv4 addresses for the last 12. And IPv6 was intentionally designed with the same layering model as IPv4—no real changes to TCP or UDP, nor any cleanup of the BSD sockets API—to ensure it would be an easy switch.

I don’t think many people are encrypting or signing large media files, independently of a filesystem (maybe excepting DRM). I have several thousand PDFs from 2020-2023 in the sub-2-megabyte range (many below 200K) where an extra megabyte could be, at least, kind of annoying. An extra megabyte in every TLS handshake, though, seems untenable. That’s got to be the most common use of network crypto; and, already, we kind of had to pull website operators kicking and screaming into the TLS world, dealing with various complaints about round-trip times; load balancing and proxies; and, yes, the handshake size. (Some of that could be cached, but basically anything cached by a browser will be used against its users for tracking; by advertisers, and, considering the Snowden context of this story, probably by other entities.)

So, I don’t know, it seems a bit far-fetched to blame the spies this time. If anything, they should want a cryptosystem that’s somewhat impractical, to ensure people will be reluctant to use it. The complexity of IPsec (including algorithm choice and negotiation) has occasionally been called an NSA conspiracy to prevent its use.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.