More Log4j News

Log4j is being exploited by all sorts of attackers, all over the Internet:

At that point it was reported that there were over 100 attempts to exploit the vulnerability every minute. “Since we started to implement our protection we prevented over 1,272,000 attempts to allocate the vulnerability, over 46% of those attempts were made by known malicious groups,” said cybersecurity company Check Point.

And according to Check Point, attackers have now attempted to exploit the flaw on over 40% of global networks.

And a second vulnerability was found, in the patch for the first vulnerability. This is likely not to be the last.

Posted on December 16, 2021 at 9:50 AM28 Comments

Comments

lurker December 16, 2021 11:29 AM

@AL
One of the threatpost links from yesterday listed attacks:
4000 from Russia
300 from China
1700 from USA

but we couldn’t expect MS to know anything about those last ones…

C U Anon December 16, 2021 11:51 AM

@ AL,

I forgot about Turkey.

Which turkey?

Hopefully there will be a lot lined qued up for the chop before the end of the month…

forgery December 16, 2021 11:59 AM

We see so many of these attacks coming from cloud providers like AWS and Azure. It’s terribly frustrating that cloud companies provide the bad guys with unlimited IP addresses from which to attack us.

Clive Robinson December 16, 2021 11:59 AM

@ Bruce, ALL,

And a second vulnerability was found, in the patch for the first vulnerability.

Who’s up for a game of dominoes to celebrate the longest nights?

Rumour has it the heat is rising in the server room, as people have a rush on for mints, as not every one wants chocolate coins for Xmas…

Marcantonio Rendino December 16, 2021 12:49 PM

It’s a small thing – though important: Please consider editing that opening to be more accurate / precise – attackers are exploiting vulnerabilities in log4j, not log4j itself. Thx

Brad December 16, 2021 1:23 PM

If you run Linux or Windows, you can use Mandatory Access Control (MAC) in enforcing mode to protect Internet facing processes from log4j and other 0-day attacks. I typically use Tomoyo on Linux. It’s built into the kernel an is the most user-friendly MAC I have ever used: https://tomoyo.osdn.jp/

UK December 16, 2021 3:38 PM

In 2017 already log4j CVSS 9.8 score:
https://nvd.nist.gov/vuln/detail/CVE-2017-5645
(and in 2019 also 9.8-> https://nvd.nist.gov/vuln/detail/CVE-2019-17571https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&query=log4j&search_type=all&isCpeNameSearch=false)
Found via https://www.theregister.co.uk/2018/01/16/oracle_quarterly_patches_jan_2018/
..at that time I saved this link because of the Spectre bug in Oracles SPARCv9
“Oracle’s Communications apps have five 9.8-rated bugs, but all are in Apache software rather than Oracle’s own efforts. Indeed, Apache Log4j appears 21 times in Oracle’s list, making CVE-2017-5645 responsible for almost ten per cent of Big Red’s patch packet.”

EvilKiru December 16, 2021 4:14 PM

Is it ONLY Log4j 2 that is vulnerable or is Log4j 1.2.16 also vulnerable? Because I’ve read contradictory things on Twitter regarding Log4j 1.x vs 2.x.

Clive Robinson December 16, 2021 4:15 PM

@ William Entriken, ALL,

I have also reported another vulnerability

And so the dominoes keep toppling…

Anyone “making book” on if it will still be toppling after the US Gov deadline of Christmas Eve latest?

Or are people going to have to climb out of the trenches singing “Silent Night” or playing the bag pipes[1].

[1] The famed story of the unoficial Christmas night ceasefire in WWI. Apparantly it started with a Minister of a Scottish regiment playing a tune on the pipes about home[2].

[2] This should not be possible on bagpipes, but… As it’s that season may this raise a smile,

https://m.youtube.com/watch?v=1iSDshYN2WY

Pete Forman December 16, 2021 5:04 PM

@EvilKiru only Log4j2 is vulnerable to the nastiest CVE-2021-44228 exploit Log4Shell using JNDI and LDAP. CVE-2021-45046 has further fixes for that. The code feature was new in v2, v1 is unaffected.

Log4j 1.2 is vulnerable to CVE-2019-17571 which leverages SocketServer. It is less of a risk because you need to do some rather unsound things to enable the hole. That’s not to say no one has. It was fixed in the corresponding CVE-2017-5645 in Log4j2 but the advice for v1 was that it went EOL in 2015, move to v2 if you want a fix.

SpaceLifeForm December 16, 2021 5:16 PM

@ EvilKiru

Exactly the same, but different.

log4j 1.x required that one activates their own footgun. Using JMSAppender.

https://www.kb.cert.org/vuls/id/930724

Interestingly, Like log4j 2.16 completely ripping out the JNDI interface, there is the same now for 2.12.2 which allows orgs that are currently stuck on Java 7 to patch without requiring Java 8.

SpaceLifeForm December 17, 2021 7:05 PM

Dependency Hell 101

You could have skipped out of this free [Java] Class. Too late. There were no prerequisites, it was a free course, but now you get to pay the tuition.

https://therecord.media/google-more-than-35000-java-packages-impacted-by-log4j-vulnerabilities/

Google’s open-source team said they scanned Maven Central, today’s largest Java package repository, and found that 35,863 Java packages use vulnerable versions of the Apache Log4j library.

This includes Java packages that use Log4j versions vulnerable to the original Log4Shell exploit (CVE-2021-44228) and a second remote code execution bug discovered in the Log4Shell patch (CVE-2021-45046).

Clive Robinson December 18, 2021 7:39 AM

@ ALL,

And still the dominoes keep falling…

People are starting to talk about,

“Mega Corp stealing code from Free Open Source”

But that’s kind of hiding the point, because the,

“Thousand eyes”

Issue will be brought up next and then the arguments will just turn into flame-wars and the real problems will not get solved…

I know some people will want to shoot me down for what I’m going to say, as they have tried to do it before and well failed in their over emotive arguments…

As I’ve said before “software engineering” as a meaningfull expression is a joke because the reality is outside of a few niche markets,

“There is no engineering in commercial Software application development.”

It’s not even “artisanal” in most cases either. The reality is it’s “Bodge it DIY”, not even upto “DIY 101” standards, or it’s “No time allowed in the deadlines”…

Most developers within a relatively short time get past “DIY 101” level in their own code otherwise they would not be being paid. So “No time allowed…”.

Which is one major reason most developers do not use their own code… Nor for that matter read and understand the code they do use. They simoly “plumb it in”. We know this, it’s why the XKCD “Dependency” cartoon exists.

That is developers either download a library or they “cut-n-past” from some online repository of “example code” of which there are many, and nearly all are terminally deficient.

Why are they terminally deficient?

Because if you look at “example code” the thing that is missing is “error and exception handeling”.

That is the assumption all input has been validated on the left (input) and there will be no issues to come back from the right (output). So just show barest functional code to demonstrate usage…

But this problem goes deeper, it’s built into the majority of multi-tasking commercial OS’s. For them to work the basic assumption is every thing flows from left to right, and if anything happens at the “output” the OS “blocks the execution thread” and goes off and does other things untill the issue at the output has cleared, or it times out and kills the thread…

It comes as a realy big shock to many developers when they have “first contact” with RTOS work. Most get Banjanxed and can not get into the required thinking process for quite some time. Some never can, and it’s an issue that is only going to get worse with the very necessary rise of “Parallel Processing”.

In Commercial OS’s you sometimes “HAVE TO” write bi-directional code, the classic example being a “Terminal Emulator” or similar communications code. Few get even close on their first attempt and for years POSIX just pretended the problem did not realy exist…

The solution is tucked away deep in the OS, the “Oh so, way to slow” way is via “Signals”. Back when PC’s ran single user the solution was a “polling loop” the user code just sat their spinning, occupining all the CPU, fast but realy inefficient.

The correct solution is “easy to say not so easy to do” it’s “Use hardware interupts and fast IO drivers to a cascade of ever larger buffers”.

In essence you split the problem into layers, at the bottom is fast very short IO drivers that copy to/from short circular buffers on hardware interupts. These circular buffersset flags that in turn are dealt with by time based “round robin” IO handlers that copy to/from the short circular buffers to much longer kernel IO buffers, that on a priority then round robin basis get pushed into user buffers and it’s here that signals and blocking usually are handled… But two things of consequence happen,

1, This process is slow but mostly does not matter as by far the majority of code used not to be bidirectional, just “gated”.

2, The bidirectional streams are now segregated and thus can in most minds be both considerd left to right.

The first is why we are hearing more about “user land IO for through put”…

The second is why just about everything gets fitted in the from “left to right” model. And this model is the underlying cause of this issue…

As I’ve previously noted it causes developers to push error trapping as far to the left as they can, and business logic as far to the right as they can…

But “What of exceptions?”… Well many say to themselves “they are so rare they never realy happen” well they do, and that is when the OS raises a signal… As most commercial application developers either don’t or can’t go there… Then you get the OS hitting what were once just called sig_abort or sig_abend[1] and turning it’s toes up. Back in the older MS-DOS/Windows days that gave the “Blue Screen of Death” if you were lucky otherwise it hung up or occasionaly went nuts or turned “Old Smokie” on you[2].

So there is an almost endemic mind set in both commercial and OSS application developers that all “errors will be stopped at the left” and “no exceptions will come from the right”.

Yes it saves a lot of code, thus time, but then things do go wrong…

Which is why right now people are going “Whaily whaily me” over log4j.

But what of next week? Next month? Next year?

Well there is a whole almost endless procession of “truck loads” of such problems comming down the road.

All entirely predictable, all entirely preventable, and none should happen, but they will… Because,

1, Developers don’t engineer the code…
2, Because managment don’t want them to…

For the first to change, managment has to let it happen, only they won’t. Because “First to Market” is the Commercial Software Industry mantra… Which gives us the ICTsec Industry as a rather unplesant symptom, not the cure.

[1] Back in the early “big iron” days signals were wild and deadly in most cases or could inflict wounds in storage etc that were mortal. IBM came up with sig_abort and sig_sabend and sig_uabend. But these were considered insufficient as resources improved. These days we have the likes of POSIX signals in their many varieties and it can take a while to get your head around them,

https://en.m.wikipedia.org/wiki/Unix_signal#POSIX_signals

My advice, take your time and importantly draw diagrams to help your thought processes.

[2] Some earlier computers could actually blow up see what was one called “poke to die”[3] and similar with 8bit home computers in the early 1980’s. One example was a casset motor controller that was driven by a “bridge driver” that alowed it to be forward and reversed. To save money / hardware / PCB space there was no hardware lockout prevention, so it was possible to turn on the bridge transistors and put a short on the power supply…

[3] Back in 8 bit home computing you had two high level programing languages, BASIC and very rarely Forth. Everything else was done in assembler… To assist in doing assembler, BASIC had two instructions PEEK that read a byte from a memory address, and POKE that wrote a byte to a memory address. Back then most IO was “memory mapped” within the 64k byte range of a 16bit address. So if you had dangerous hardware and some kiddy wrote a loop that “Poked Memory” address after address like a game of Russian Roulette sometimes they would get lucky sometimes they would not.

JonKnowsNothing December 18, 2021 2:16 PM

@ Clive, @ ALL

While your analysis covers 99.99% of the cases it misses that the vast majority of programmers/software engineers and even program designers are not the ones in charge of “security”.

Security remains a specialty niche where “someone else is doing that”.

There are several items on the worktable:

  1. You are tasked to make X
  2. X fits within Y product line
  3. Y product line is/was/has a “frozen design” state

If you do not know how to make X, your find someone who will give you a cut n paste version that you can adapt to your needs. Perhaps there are some that come out of a box-kit that can be used as-is, but that was not my experience.

I am thankful for the hand-up of “sample code” or “sample program” which mostly consisted of scrounging the source code library because another programmer pointed out that “there was something in there that did similar”.

The harmful part(s) are when the code underneath is changed and there isn’t any notes to that extent. Rarely does anyone put notes in code. (1)

The even more harmful part is when slip-ins are placed by NoTNicePersons and no one notices because the code compiles and regression testing has no failures.

There is zero chance that I would ever compete against a 3L-Insertion into the code base. I’ve captured my share of OhShyte bugs but none of them were 3LPlus types.

I can bolt the doors, lock the windows, and secure the premises in my room, but I cannot protect the next rooms. Nor can I protect against termites.

===
1. RL tl;dr

  • I put a note in 2 sections of code indicating that any change to these sections needed to be mirrored in the other. Sure ’nuff someone made a change in one section but not the other. The fail-report landed on my desk, the someone said they never paid attention to the note because their task did not call out the mirror side. They got A but not A&B. So that’s what they did – just A.

Security Sam December 19, 2021 1:42 PM

The more novel security controls evolve
The more they stay more or less the same
While all the ‘chicken littles’ with resolve
Instead of bad code prefer Log4j to blame

Paul December 19, 2021 5:23 PM

Remember that the exploit is only applicable if the logging implementation that the running application uses is log4j v2. Why is this an important distinction? If I use a library that uses log4j2 in its classes, but my application runs with Logback or even log4j version 1, then my application is NOT at risk, as the risk doesn’t come from the API usage, but the runtime engine.

Furthermore, log4j 2 is actually not THAT popular in enterprises. I work in a very large investment back and the massive bulk of our apps are run using Logback . The older apps (generally those running from the java7 or before era!) use log4j version 1. Logback took over the mantle from log4j version 1 and the new(ish) log4j version 2 has failed to capture real market share as it is (as they decided to not use the new Java SLF which annoyed a lot of developers). Only a few apps that like the lambda support actually went for it. As such, in my bank we have very little to patch (but of course this is being taken VERY seriously and is the worst breach I’ve heard of in years due to questionable goals by the log4j2 team)

I could find over 1k lines of code that uses log4j2s logger API in our banks repos – but when we check them all in detail, nearly all were using logback as the provider with routing libraries added to route log4j2 API to logbacks logging engine.

If this were a logback vulnerability or even log4j version 1 on the same level as this one, then most of the enterprise world would be exposed!

Clive Robinson December 19, 2021 5:48 PM

@ Paul,

(but of course this is being taken VERY seriously and is the worst breach I’ve heard of in years due to questionable goals by the log4j2 team)

The thing is that those “questionable goals” are exactly the same ones driving the entire consumer/commercial software application market of,

Maintaining Sales by,

1, Appearing to stay relevant.
2, Trying to be all things to all men.

The result is “bloat” and attendent “insecurity” from,

3, Ease of use prioritization.

We have seen this since before Microsoft got going back more than four decades ago and made it their basic development model…

All the Internet has added is,

4, Testing before release can be minimized as patching is easy.

All you are realy buying when you get down to it is,

5, A limited time access ticket.

Which is rapidly evolving into,

6, “Software as a Service”(SaaS) on “Cloud Servers”.

So enabeling,

7, massive theft of “information”.

That can then be,

6, Repackaged and sold to others.

Need I go on?

SpaceLifeForm December 22, 2021 3:41 AM

Log4j Scanner

If you are smart, you do not abuse.

This is for your own org, or to check others with permission.

hxtps://github.com/cisagov/log4j-scanner

This repository provides a scanning solution for the log4j Remote Code Execution vulnerabilities (CVE-2021-44228 & CVE-2021-45046). The information and code in this repository is provided “as is” and was assembled with the help of the open-source community and updated by CISA through collaboration with the broader cybersecurity community.

SpaceLifeForm December 22, 2021 5:31 PM

There are other dots

hxtps://www.scmp.com/tech/big-tech/article/3160670/apache-log4j-bug-chinas-industry-ministry-pulls-support-alibaba-cloud

Apache Log4j bug: China’s industry ministry pulls support from Alibaba Cloud for not reporting flaw to government first

ResearcherZero December 23, 2021 4:35 AM

Parts of the Belgian Defense Ministry’s computer networks have been down since Thursday after a cyber incident in which attackers exploited the Apache Log4j vulnerability, government officials said.

“All weekend our teams have been mobilized to control the problem, continue our activities and warn our partners,”

“The priority is to keep the network operational. We will continue to monitor the situation.”

The Belgian Defense Ministry is the first reported high-profile government victim of the vulnerability, but unlikely to be the last given the ubiquity of Log4j in a host of enterprise software popular in the public and private sector.

Affected parts of the Belgian network were segmented after the attack was discovered, Séverin says. Systems including email appear to still be down as of Monday morning.

The Belgian government has not attributed the attack to any group or nation-state.
https://www.vrt.be/vrtnws/nl/2021/12/20/defensie-slachtoffer-zware-cyberaanval-deel-netwerk-al-dagen-pl/

aggie January 6, 2022 12:33 PM

“And a second vulnerability was found, in the patch for the first vulnerability.”

You normally only see this kind of incompetence at the government level.

john January 6, 2022 12:35 PM

“It’s terribly frustrating that cloud companies provide the bad guys with unlimited IP addresses from which to attack us.”

It’s so much easier to “blame the infrastructure” (almost as good as “blame the victim”) instead of blaming the model of software development that doesn’t give a damn about security until after the fact.

Security costs money, and that’s why corporations don’t want to pay for it. Unless they can get some cheap H1-Bs to do it, that is.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.