Schneier on Security

Clive Robinson • June 21, 2021 7:39 AM

@ Bruce, ALL,

The four conclusions are almost the same as have been given for computers since the 1960’s

Indicating that perhaps on “Machine Learning” there is nothing new above ordinary software.

Or to put it another Way,

“Man thinks, Man code, code runs faster, code runs finer, but code does not do anything not thought up by man.”

However there should be another conclusion,

“Speed kills”

If man offloads skills to computers, then yes those determanistic skills can be done faster, more effectively, and more efficiently by a computer.

But what of non determanistic skills?

It’s easy to see how, learning is killed, inovation is killed, and thus is progress starved and killed.

We half hartedly joke about “eveloutionary cul-de-sacs” and saber tooth togers. But in all such things there is a germ of truth.

Is over reliance on machine learning going to be mankinds evolutionary cul-de-sac?

We are already seeing issues with machine learning and the justice system. Where the non transparancy of so called neural networks and similar that are little more than glorified statistics packages are being used by authoritarian guard labour to evad responsability for their desired actions by hiding behind “The Computer Says” excuse.

We know the GIGO principle applies by the trash trailer full with machine learning. By running “hidden tests” criteria can be selected to give desired outcomes. That is the criteria though seeming random will give rise to a “training data set” that poisons or predisposes the ML system and causes it to adopt certain desired characteristics that are effectively automated “isms”.

We further know that the likes of Peter Thiel of Palantir are pushing machine learning systems into law enforcment. The hidden aim of which is “dependency thus profit” the same as drug dealers do, they sell you junk cheap, and you become dependent then thay jack the price.

In the Palantir model the aim is to get systems in, and get detectives phased out, thus the money that was spent on detectives goes to Geoff Thiel and Co, not on bringing detectives forward and new ones to follow them.

So the Machine Learning, which is incapable of learning and thinking thus responding in an intelligent way to changes in criminal activities, ceases to move with the criminal threat, but by the time that is realised the continuity of human detectives is broken, thus much of the most important skills are lost for quite some time if not for good. Thus irreparable harm is done for short term gain. It might be a politicians dream, but it will be societies near endless nightmare.

Clive Robinson • June 22, 2021 6:23 AM

@ JPA,

…issues with “machine learning”…

Such linguistic issues should not be an issue, but clearly are.

Back in the 1980’s I was “playing” not just with toy robots driven by 8bit microcontrolers as research demonstrators but with bl@@dy great Puma industrial robots used in the Auto Industry driven by Micro Vaxes and other high end 32bit computers runing Real Time Opetating Systems(RTOS). Such “robots” would smash you skull to pieces and not even be effected due to their basic lack of resources.

We would not recognise them as robots these days just more flexible tool extentions to other tools like CNC machines.

Back then everything was very very very mechanistically determined. For instance it would pick a car door up off of a stack, not because it could sense where the pickup point was, but because it was given the exact point, exact orientation and exact route to take to reach it.

Re-calibration was done by having a sensor in a safe place the arm would traverse through and the extetnal sensor would provide correction data to the control program.

That is what humans do for gross motor control, but human fine motor control is done by sensors in the fingers that can be a thousand or so times better than our eyes are at a meter distance.

Engineers know these things almost implicitly and are rarely confused. In part because the language used is very specific within the domain and engineers know from context when domain specific language is being used and when more general 20,000ft view sloppy language is being used to convey generalised meaning.

Working with many “programmers” they tend to just be sloppy with language unless it is very specifically source code on the screen. Every thing else such as flow diagrams entity descriptions all sloppy and arm wavery unless they are domain experts in other fields such as researchers, scientists, mathmeticians engineers that use software as a tool like they would a screwdriver, drill, pipette or equation.

Unfortunately there are new researchers that in effect have “soft systems” backgrounds or are even further away from what arectraditionaly called the “hard sciences”. They kind of fill the gap that architrcts and fashion designers do in their respective domains. That is they take a conceptual approach where fluidity of language is high and constantly changing. You see this in some “web designers” and similar dealing with the look and feal concepts of the HCI.

Yes it has it’s place but there is to much “bleed through” and even researchers are now succumbing to conceptulising and soft abstractions are seen by some as almost the “be all and end all”…

This leads to “soft explanations” that give rise to incorect notions.

Engineers who work with few soft abstractions, tend to have an implicit horror of them, unsuprisingly they know what 10,000 Newtons force[1] actually means in very real terms. As many others do not the result is they can get written off as the equivalent of “Mansplainers” by those who fail to realise the very real danger they are failing to see.

[1] For those unfamiliar with Newtons of force, there is a very cute way to think about it. Sit there imagin you are Sir Isasac with your hand out stretched palm up with a quater pound English eating apple wresting on it – that is roughly one Newton of force. Now imagine trying to hold ten thousand apples all in your palm at the same time… What do you imagine would happen to your wrist, elbow etc if you tried? An engineer could Mansplain that as crudely[2] as you like, but could an architect or fashion desiger do it conceptualy?

[2] Think Michael Cain and his oft quoted line about doors… The BMC Mini that featured promenently in that film would –if memory serves correctly– be, if you could pick it up and get it on one hand, about 5,000 Newtons (conceptual enough 😉

Clive Robinson • June 22, 2021 6:42 AM

@ Winter,

Or a “ton up” for those not using those strange French revolutionary measures 😉

Clive Robinson • July 3, 2021 6:41 PM

@ A Nonny Bunny,

Actually, pretty much all code does things not thought up by man, because man didn’t really think through what man coded. Bugs aplenty.

That is usually down to the fact that the “Man thinks, Man code” is not done sufficiently iteratively.

Thus the “code runs faster, code runs finer” iterative process automatically becomes a failed process.

Broadly you could argue one or more of these are to blaim,

1, Man is deficient
2, Process is deficient
3, Process is constrained / curtailed.

I’d argue all three are in part to blaim in all “bugs” and most definitely in all failed projects.

But you raise,

You can also generate code using genetic algorithms and a true number generator and get code not thought up by man at all.

I think either you are just restating the problem differently trying to make it look like it’s a different process (which is fairly easy to prove it is not). Or you’ve left something out of your statment…

That which is missing being something Alan Turing was fairly insistant went into the earliest of computers, as part of the process. It’s something I use fairly frequently and it comes under the more general term of “Extended Monte Carlo Methods”(EMCM).

The problem with any EMCM is it almost always falls into one of the “Arthur C Clarke’s” eponymous cognative failing bear traps by those observing such a process[1].

Whilst Clarke did not call them “cognative failing” they appeared in an essay with “Hazards of Prophecy: The Failure of Imagination” which is essentially saying the same.

The usual bear trap into which you have wandered is,

“Any sufficiently advanced technology is indistinguishable from magic.”

Which has become more pertinent in the past few years as,

“Any technology, no matter how primitive, is magic to those who don’t understand it.”

And from that, why we have the version for ML of,

“Any sufficiently advanced act of benevolence is indistinguishable from malevolence”

Which is the “utopian fear” of “The machines taking over” and “making us their pets” or similar. Great for “selling copy” to the “Woo Woo Crystal Healing Croud” but seriously? Long answer short NO. But it also gets the real “Chicken Little” veneer with the notion of “Sky Net” from the “Terminator” films… So is an almost guarenteed sell…

The notion of taking “random” and imbuing it with mystical power, is a failing we more normally see in gamblers with their belief in “lucky streaks”. Or those who have “It’s my turn” syndrome because somebody came up with the phrase “You have to be in it to win it” to gull them out of money. When simple mathmatics tells you, you will probably not win it in your life time.

Even the simple study of “Brownian motion” in high school should tell people a lot about “random” and how “perceived effects” are actually caused by less obvious “underlying causes” that can work either way[2] apparently unpredictably.

Most humans have a failing which is an artifact of a survival trait. We see meaning in cloud shapes, ink blots, shadows, and shading, that are realy not there. It is the result of a trade off made long ago between expending energy responding rapidly or being taken for lunch. Thus the acceptance that seeing something and scooting up a tree etc falsely because it might be a preditor stalking, was better than being certain but then not able to make it to the tree and becoming a “dogs breakfast”.

The problem is it can become an irrational way of living often called “follow your hunches”. People glorify the very low probability successes and ignore the high probability failures. We even have reality TV shows like “Dragons Den” and the “Apprentice” pushing such ideas to those who can not reason that the shows “hook” is in fact it’s a “freak show” for those that get their rocks off on others failings.

You talk of “generate code using genetic algorithms” as some kind of “magic” not a creation of “man’s thinking” or you think although you don’t say it “random” is magic when it’s not.

A thought for you, there are “lotto games” around with the worked out odds of winning being 1 in ~15million and around ~60million individual tickets purchased each week. So what are the odds of the top prize not being won in any given week? How about every four weeks or longer?

Random algorithms frequently trade the potential of being faster to a solution against the probability of never finding a solution. Where as a determanistic search may on average be slower, it does get to finish in a known time period. Random algorithms can be even faster when you only need a solution within some percentage of the optimal solution.

Yes you can use “random” for other things but there is no inteligence or magic behind random, and it’s use from generation onwards is still determanistic and a product of “mans thinking” good or bad.

[1] Also called “Clarke’s Three Laws” whilst at first they appear general the first and third are about cognative failings in “outsider observers” whilst the second is more about the cognative failings of “insider observers”.

https://en.m.wikipedia.org/wiki/Clarke%27s_three_laws

Hence Isaac Asimov’s Corollary to Clarke’s First Law, effectively invokes Clarke’s the third law and humanities fondness of ascribing power to that they do not comprehend. A fondness that unfortunately goes both ways and shows my usual points that,

1.1 Technology is agnostic to use.
1.2 It is the Directing mind that decides the use.
1.3 It is the Observing mind that decides if the use is good or bad.

Clarke’s second law is an impricise way of saying another of my oft repeated points about judging technology,

1.4 If the laws of physics alow it.
1.5 Then someone will eventually try to build it.

The fact that over the past century and a half, people have made what are different facet arguments to essentially the same gem of an idea should give you an indication of just how fundemental to modern society the rationalism of science is compared to the mystercism engendered for societal control in the past.

[2] Having had the misfortune to describe the issues around “pivot points” and “finite precision” and why “true” inverse matrices are almost always “staged examples”, to students and other neophytes thinking to make use of them to solve real world problems with computers. With them not realising the reality of results being “not just off a little”, but “chasing Voyager”, unless a lot of corrective techniques are used which tend to make the “cure” often worse than the “disease” especially in constrained environments.