I use non-Unicode mode in the terminal emulator (and text editors, etc), I use a non-Unicode locale, and will always use ASCII for most kind of source code files (mainly C) (in some cases, other character sets will be used such as PC character set, but usually it will be ASCII). Doing this will mitigate many of this when maintaining your own software. I am apparently not the only one; I have seen others suggest similar things. (If you need non-ASCII text (e.g. for documentation) you might store them in separate files instead. If you only need a small number of them in a few string literals, then you might use the \x escapes; add comments if necessary to explain it.)

The article is about in JavaScript, although it can apply to other programming languages as well. However, even in JavaScript, you can use \u escapes in place of the non-ASCII characters. (One of my ideas in a programming language design intended to be better instead of C, is that it forces visible ASCII (and a few control characters, with some restrictions on their use), unless you specify by a directive or switch that you want to allow non-ASCII bytes.)

IMO while the bar is high to say "it's the responsibility of the repository operator itself to guard against a certain class of attack" - I think this qualifies. The same way GitHub provides Secret Scanning [0], it should alert upon spans of zero-width characters that are not used in a linguistically standard way (don't need an LLM for this, just n-tuples).

Sure, third-party services like the OP can provide bots that can scan. But if you create an ecosystem in which PRs can be submitted by threat actors, part of your commitment to the community should be to provide visibility into attacks that cannot be seen by the naked eye, and make that protection the norm rather than the exception.

[0] https://docs.github.com/en/get-started/learning-about-github...

• andrewflnr

  today at 4:44 PM

  Regardless of the thorny question of whether it's Github's responsibility, it sure would be a good thing for them to do ASAP.

  • godelski

    today at 6:51 PM

    Here's the big reason GitHub should do it:

      It makes the product better
    

    I know people love to talk money and costs and "value", but HN is a space for developers, not the business people. Our primary concern, as developers, is to make the product better. The business people need us to make the product better, keep the company growing, and beat out the competition. We need them to keep us from fixating on things that are useful but low priority and ensuring we keep having money. The contention between us is good, it keeps balance. It even ensures things keep getting better even if an effective monopoly forms as they still need us, the developers, to make the company continue growing (look at monopolies people aren't angry at and how they're different). And they need us more than we need them.

    So I'd argue it's the responsibility of the developers, hired by GitHub, to create this feature because it makes the product better. Because that's the thing you've been hired for: to make the product better. Your concern isn't about the money, your concern is about the product. That's what you're hired for.

    • tapland

      today at 6:53 PM

      Tldr: Yeah it would make it better!

      • godelski

        today at 7:16 PM

        I hope I left the lead as the lead.

        But I also think we've had a culture shift that's hurting our field. Where engineers are arguing about if we should implement certain features based on the monetary value (which are all fictional anyways). But that's not our job. At best, it's the job of the engineering manager to convince the business people that it has not only utility value, but monetary.

  • jacquesm

    today at 5:23 PM

    It absolutely is. They are simply spreading malware. You can't claim to be a 'dumb pipe' when your whole reason for existence is to make something people deemed 'too complex' simple enough for others to use, then you have an immediate responsibility to not only reduce complexity but to also ensure safety. Dumbing stuff down comes with a duty of care.

• zzo38computer

  today at 7:13 PM

  I think a "force visible ASCII for files whose names match a specific pattern" mode would be a simple thing to help. (You might be able to use the "encoding" command in the .gitattributes file for this, although I don't know if this would cause errors or warnings to be reported, and it might depend on the implementation.)

It baffles me that any maintainer would merge code like the one highlighted in the issue, without knowing what it does. That’s regardless of being or not being able to see the “invisible” characters. There’s a transforming function here and an eval() call.

The mere fact that a software maintainer would merge code without knowing what it does says more about the terrible state of software.

I feel like the threat of this type of thing is really overstated.

Sure the payload is invisible (although tbh im surprised it is. PUA characters usually show up as boxes with hexcodes for me), but the part where you put an "empty" string through eval isn't.

If you are not reviewing your code enough to notice something as non sensical as eval() an empty string, would you really notice the non obfuscated payload either?

The `eval` alone should be enough of a red flag

• jeltz

  today at 5:27 PM

  Yeah, I would have loved to see an example where it was not obvious that there is an exploit. Where it would be possible for a reviewer to actually miss it.

• godelski

  today at 7:08 PM

  I'm not a JS person, but taking the line at face value shouldn't it to nothing? Which, if I understand correctly, should never be merged. Why would you merge no-ops?

• kordlessagain

  today at 4:18 PM

  No it’s not.

  • simonreiff

    today at 4:52 PM

    OWASP disagrees: See https://cheatsheetseries.owasp.org/cheatsheets/Nodejs_Securi..., listing `eval()` first in its small list of examples of "JavaScript functions that are dangerous and should only be used where necessary or unavoidable". I'm unaware of any such uses, myself. I can't think of any scenario where I couldn't get what I wanted by using some combination of `vm`, the `Function` constructor, and a safe wrapper around `JSON.parse()` to do anything I might have considered doing unsafely with `eval()`. Yes, `eval()` is a blatant red flag and definitely should be avoided.

  • jacquesm

    today at 5:24 PM

    While there are valid use cases for eval they are so rare that it should be disabled by default and strongly discouraged as a pattern. Only in very rare cases is eval the right choice and even then it will be fraught with risk.

  • godelski

    today at 7:11 PM

    The parent didn't say "there's no legitimate uses of eval", they said "using eval should make people pay more attention." A red flag is a warning. An alert. Not a signal saying "this is 100% no doubt malicious code."

    Yes, it's a red flag. Yes, there's legitimate uses. Yes, you should always interrogate evals more closely. All these are true

  • pavel_lishin

    today at 4:27 PM

    When is an eval not at least a security "code smell"?

  • SahAssar

    today at 4:27 PM

    It really is. There are very few proper use-cases for eval.

    • nswango

      today at 5:36 PM

      For a long time the standard way of loading JSON was using eval.

      • bawolff

        today at 7:52 PM

        Not that long, browsers implemented JSON.parse() back in 2009. JSON was only invented back in 2001 and took a while to become popular. It was a very short window more than a decade ago when eval made sense here.

        Eval for json also lead to other security issues like XSSI.

      • _flux

        today at 6:04 PM

        And why do we not anymore make use of it, but instead implemented separate JSON loading functionality in JavaScript? Can you think of any reasons beyond performance?

        • bawolff

          today at 7:47 PM

          I'd be surprised if there is a performance benefit of processing json with eval(). Browsers optimize the heck out of JSON.

        • bulbar

          today at 6:42 PM

          Why did you opt in for such a comment while a straight forward response without belittling tone would have achieved the same?

          • _flux

            today at 6:47 PM

            I actually gave it some thought. I had written the actual reason first, but I realized that the person I was responding to must know this, yet keeps arguing in that eval is just fine.

            I would say they are arguing that in bad faith, so I wanted to enter a dialogue where they are either forced to agree, or more likely, not respond at all.

Small discussion yesterday (9+9 points, 9+4 comments) https://news.ycombinator.com/item?id=47374479 https://news.ycombinator.com/item?id=47385244

Unicode should be for visible characters. Invisible characters are an abomination. So are ways to hide text by using Unicode so-called "characters" to cause the cursor to go backwards.

Things that vanish on a printout should not be in Unicode.

Remove them from Unicode.

• pvillano

  today at 5:34 PM

  Unicode is "designed to support the use of text in all of the world's writing systems that can be digitized"

  Unicode needs tab, space, form feed, and carriage return.

  Unicode needs U+200E LEFT-TO-RIGHT MARK and U+200F RIGHT-TO-LEFT MARK to switch between left-to-right and right-to-left languages.

  Unicode needs U+115F HANGUL CHOSEONG FILLER and U+1160 HANGUL JUNGSEONG FILLER to typeset Korean.

  Unicode needs U+200C ZERO WIDTH NON-JOINER to encode that two characters should not be connected by a ligature.

  Unicode needs U+200B ZERO WIDTH SPACE to indicate a word break opportunity without actually inserting a visible space.

  Unicode needs MONGOLIAN FREE VARIATION SELECTORs to encode the traditional Mongolian alphabet.

  • WalterBright

    today at 6:29 PM

    > Unicode needs tab, space, form feed, and carriage return.

    Those are legacied in with ASCII. And only space and newline are needed. Before I check in code to git, I run a program that removes the tabs and linefeeds.

    > Unicode needs U+200E LEFT-TO-RIGHT MARK and U+200F RIGHT-TO-LEFT MARK to switch between left-to-right and right-to-left languages.

    !!tfel ot thgir ,am ,kooL

    > Unicode needs U+115F HANGUL CHOSEONG FILLER and U+1160 HANGUL JUNGSEONG FILLER to typeset Korean.

    I don't believe it.

    > Unicode needs U+200C ZERO WIDTH NON-JOINER to encode that two characters should not be connected by a ligature.

    Not needed.

    > Unicode needs U+200B ZERO WIDTH SPACE to indicate a word break opportunity without actually inserting a visible space.

    How on earth did people read printed matter without that?

    > Unicode needs MONGOLIAN FREE VARIATION SELECTORs to encode the traditional Mongolian alphabet.

    Somehow people didn't need invisible characters when printing books.

    • bulbar

      today at 6:50 PM

      That's a very narrow view of the world. One example: In the past I have handled bilingual english-arabic files with switches within the same line and Arabic is written from left to right.

      There are also languages that are written from to to bottom.

      Unicode is not exclusively for coding, to the contrary, pretty sure it's only a small fraction of how Unicode is used.

      > Somehow people didn't need invisible characters when printing books.

      They didn't need computers either so "was seemingly not needed in the past" is not a good argument.

    • jmusall

      today at 7:15 PM

      The fact is that there were so many character sets in use before Unicode because all these things were needed or at least wanted by a lot of people. Here's a great blog post by Nikita Prokopov about it: https://tonsky.me/blog/unicode/

    • WalterBright

      today at 6:44 PM

          Look Ma
          xt! N !
          e tee S
          T larip
      

      (No Unicode needed.)

    • chongli

      today at 6:56 PM

      Unicode is for human beings, not machines.

• tetha

  today at 7:43 PM

  That ship has sailed, but I consider Unicode a good thing, yet I consider it problematic to support Unicode in every domain.

  I should be able to use Ü as a cursed smiley in text, and many more writing systems supported by Unicode support even more funny things. That's a good thing.

  On the other hand, if technical and display file names (to GUI users) were separate, my need for crazy characters in file names, code bases and such are very limited. Lower ASCII for actual file names consumed by technical people is sufficient to me.

• bawolff

  today at 7:53 PM

  Good luck with that given there are invisible characters in ascii.

  Also this attack doesnt seem to use invisible characters just characters that dont have an assigned meaning.

• eviks

  today at 7:39 PM

  So you'd remove space and tab from Unicode?

• luke-stanley

  today at 5:54 PM

  So we need a new standard problem due to the complexity of the last standard? Isn't unicode supposed to be a superset of ASCII, which already has control characters like new space, CR, and new lines? xD

  • WalterBright

    today at 6:31 PM

    The only ones people use any more are newline and space. A tab key is fine in your editor, but it's been more or less abandoned as a character. I haven't used a form feed character since the 1970s.

• WalterBright

  today at 5:21 PM

  Another dum dum Unicode idea is having multiple code points with identical glyphs.

  Rule of thumb: two Unicode sequences that look identical when printed should consist of the same code points.

  • estebank

    today at 7:40 PM

    If anything, Unicode should have had more disambiguated characters. Han unification was a mistake, and lower case dotted Turkish i and upper case dotless Turkish I should exist so that toUpper and toLower didn't need to know/guess at a locale to work correctly.

  • nswango

    today at 5:34 PM

    So you think that the letters in the Greek and Cyrillic alphabets which are printed identically to the Latin A should not exist?

    And, for example, Greek words containing this letter should be encoded with a mix of Latin and Greek characters?

    • WalterBright

      today at 6:34 PM

      > So you think that the letters in the Greek and Cyrillic alphabets which are printed identically to the Latin A should not exist?

      Yes. Unicode should not be about semantic meaning, it should be about the visual. Like text in a book.

      > And, for example, Greek words containing this letter should be encoded with a mix of Latin and Greek characters?

      Yup. Consider a printed book. How can you tell if a letter is a Greek letter or a Latin letter?

      Those Unicode homonyms are a solution looking for a problem.

      • Yokohiii

        today at 7:09 PM

        Unicode is about semantics not appearance. If you don't need semantics then use something different.

      • Muromec

        today at 7:02 PM

        >Yup. Consider a printed book. How can you tell if a letter is a Greek letter or a Latin letter?

        I can absolutely tell Cyrillic k from the lating к and latin u from the Cyrillic и.

        >should not be about semantic meaning,

        It's always better to be able to preserve more information in a text and not less.

    • Yokohiii

      today at 7:04 PM

      What about numbers? Would they be assigned to arabic only? I guess someone will be offended by that.

      While at it we could also unify I, | and l. It's too confusing sometimes.

  • wcoenen

    today at 6:50 PM

    As far as I know, glyphs are determined by the font and rendering engine. They're not in the Unicode standard.

  • jeltz

    today at 5:32 PM

    I don't think that would help much. There are also characters which are similar but not the same and I don't think humans can spot the differences unless they are actively looking for them which most of the time people are not. If only one of two glyphs which are similar appear in the text nobody would likely notice, expectation bias will fuck you over.

    • WalterBright

      today at 6:34 PM

      I wonder how anybody got by with printed books.

• today at 6:22 PM

• moritzruth

  today at 5:14 PM

  greatidea,whoneedsspacesanyway

  • WalterBright

    today at 5:15 PM

    Spaces appear on a printout.

• uhoh-itsmaciek

  today at 5:42 PM

  >Remove them from Unicode.

  Do you honestly think this is a workable solution?

  • WalterBright

    today at 6:35 PM

    Yes, absolutely. See my other replies.

• abujazar

  today at 5:14 PM

  Invisible characters are there for visible characters to be printed correctly...

  • WalterBright

    today at 6:35 PM

    I'll grant that a space and a newline are necessary. The rest, nope.

    • abujazar

      today at 7:05 PM

      You're talking about a subset of ASCII then. Unicode is supposed to support different languages and advanced typography, for which those characters are necessary. You can't write e.g. Arabic or Hebrew without those "unnecessary" invisible characters.

Their button animations almost "crash" Firefox mobile. As soon as I reach them the entire page scrolls at single digit FPS.

Looks like the repo owner force-pushed a bad commit to replace an existing one. But then, why not forge it to maintain the existing timestamp + author, e.g. via `git commit --amend -C df8c18`?

Innocuous PR (but do note the line about "pedronauck pushed a commit that referenced this pull request last week"): https://github.com/pedronauck/reworm/pull/28

Original commit: https://github.com/pedronauck/reworm/commit/df8c18

Amended commit: https://github.com/pedronauck/reworm/commit/d50cd8

Either way, pretty clear sign that the owner's creds (and possibly an entire machine) are compromised.

• chrismorgan

  today at 5:30 PM

  The value of the technique, I suppose, is that it hides a large payload a bit better. The part you can see stinks (a bunch of magic numbers and eval), but I suppose it’s still easier to overlook than a 9000-character line of hexadecimal (if still encoded or even decoded but still encrypted) or stuff mentioning Solana and Russian timezones (I just decoded and decrypted the payload out of curiosity).

  But really, it still has to be injected after the fact. Even the most superficial code review should catch it.

  • vitus

    today at 5:48 PM

    Agreed on all those fronts. I'm just dismayed by all the comments suggesting that maintainers just merged PRs with this trojan, when the attack vector implies a more mundane form of credential compromise (and not, as the article implies, AI being used to sneak malicious changes past code review at scale).

    • jeltz

      today at 6:04 PM

      Yeah, the attack vector seems to be stolen credentials. I would be much more interested in an attack which actually uses Invisible characters as the main vector.

Attacks employing invisible characters are not a new thing. Prior efforts here include terminal escape sequences, possibly hidden with CSS that if blindly copied and pasted would execute who knows what if the particular terminal allowed escape sequences to do too much (a common feature of featuritis) or the terminal had errors in its invisible character parsing code.

For data or code hiding the Acme::Bleach Perl module is an old example though by no means the oldest example of such. This is largely irrelevant given how relevant not learning from history is for most.

Invisible characters may also cause hard to debug issues, such as lpr(1) not working for a user, who turned out to have a control character hiding in their .cshrc. Such things as hex viewers and OCD levels of attention to detail are suggested.

Why didn't some make av rule to find stuff like this, they are just plain text files

• nine_k

  today at 4:05 PM

  The rule must be very simple: any occurrence of `eval()` should be a BIG RED FLAG. It should be handled like a live bomb, which it is.

  Then, any appearance of unprintable characters should also be flagged. There are rather few legitimate uses of some zero-width characters, like ZWJ in emoji composition. Ideally all such characters should be inserted as \xNNNN escape sequences, and not literal characters.

  Simple lint rules would suffice for that, with zero AI involvement.

  • WalterBright

    today at 5:14 PM

    > There are rather few legitimate uses of some zero-width characters, like ZWJ in emoji composition.

    Emojis are another abomination that should be removed from Unicode. If you want pictures, use a gif.

    • _flux

      today at 6:09 PM

      Arguably them being in Unicode is an accessibility issue, unless we thought to standardize GIF names, and then that already sounds a lot like Unicode.

      • WalterBright

        today at 6:20 PM

        How is it an accessibility issue? HTML allows things like little gif files. I've done this myself when I wrote text that contained Egyptian hieroglyphs. It works just fine!

        • _flux

          today at 6:43 PM

          I mean if you don't have sight.

          • WalterBright

            today at 6:46 PM

            Then use words. Or tooltips (HTML supports that). I use tooltips on my web pages to support accessibility for screen readers. Unicode should not be attempting to badly reinvent HTML.

    • sghitbyabazooka

      today at 6:03 PM

      ( ꏿ ﹏ ꏿ ; )

  • hamburglar

    today at 5:30 PM

    I think there’s debate (which I don’t want to participate in) over whether or not invisible characters have their uses in Unicode. But I hope we can all agree that invisible characters have no business in code, and banishing them is reasonable.

  • trollbridge

    today at 4:25 PM

    In our repos, we have some basic stuff like ruff that runs, and that includes a hard error on any Unicode characters. We mostly did this after some un-fun times when byte order marks somehow ended up in a file and it made something fail.

    I have considered allowing a short list that does not include emojis, joining characters, and so on - basically just currency symbols, accent marks, and everything else you'd find in CP-1521 but never got around to it.

• abound

  today at 3:27 PM

  Yeah it would have been nice to end with "and here's a five-line shell script to check if your project is likely affected". But to their credit, they do have an open-source tool [1], I'm just not willing to install a big blob of JavaScript to look for vulns in my other big blobs of JavaScript

  [1] https://github.com/AikidoSec/safe-chain

  • nine_k

    today at 4:30 PM

    Something like this should work, assuming your encoding is Unicode (normally UTF-8), which grep would interpret:

      grep -P '[\x{200B}\x{200C}\x{200D}\x{FEFF}]' code.ts
    

    See https://stackoverflow.com/q/78129129/223424

I wonder if this could be used for prompt injection, if you copy and paste the seemingly empty string into an LLM does it understand? Maybe the affect Unicode characters aren’t tokenized.

Why can't code editors have a default-on feature where they show any invisible character (other than newlines)? I seem to remember Sublime doing this at least in some cases... the characters were rendered as a lozenge shape with the hex value of the character.

Is there ever a circumstance where the invisible characters are both legitimate and you as a software developer wouldn't want to see them in the source code?

eval() used to be evil....

Are people using eval() in production code?

Back in time I was on hacking forums where lot of script kiddies used to make malicious code.

I am wondering how that they've LLM, are people using them for making new kind of malicious codes more sophisticated than before?

• Yokohiii

  today at 4:56 PM

  In this case LLMs were obviously used to dress the code up as more legitimate, adding more human or project relevant noise. It's social engineering, but you leave the tedious bits to an LLM. The sophisticated part is the obscurity in the whole process, not the code.

[dead]

[dead]

[dead]

I don't have to worry about any of this.

My clawbot & other AI agents already have this figured out.

/s