Light in the other end of the tunnel ?

 Hi folks,

I hope you are all fine ... but as you all know, there is a shred of hope :-)

Long time I did not write but I still contributed here and there as usual to projects I already mentioned, some few are new.

From Barony to the usual custom memory allocators (jemalloc particularly welcomed few BSD changes), redis, memcached, minetest ... PHP 8 is scheduled to be released around the 26th. Speaking of PHP, xdebug finally accepted my fixes from months months ago :-) The vast majority of the overall contributions had been merged or at least accepted.

For first time, I did few changes in OpenSSL for BSD systems and Haiku.

Outside of contributions, the recent release of the Mac M1 line has a good chance to interest me, professionally speaking primarily, with its iOs and Intel compatibility layer. We shall see.

Nothing too special apart of this, during this lockdown, I took the opportunity to go back to one of my teenage hobby and went to (re)record this little piece :-)

Fingers crossed things will go better in 2021, in the meantime I wish you all well ;-)

Autumn falls, time for changes

Hi dear readers,

I am glad to say now LLVM had fully moved to the git workflow since the last post ! For now there is the next logical remaining concern which is do we handle Pull requests and bug tracking as it is now and do we move fully to GitHub ? We shall see how it goes, it is not so obvious to go in one direction or another, Phabricator having nice code commenting feature for example but hard to maintain over time in another hand (Phabricator for LLVM is actually a fork). Either way, it is already nice to be where we are...

While we re into LLVM, I have spent little time, during the svn to git transition, to add basic OpenBSD support for the OpenMP api, knowing this os cannot support the cpu affinity part.
Still spending time also with the NodeJS community, aside of the usual review, I try to push upstream some FreeBSD patches from the related port to diminish its "technical debt", for instance here, more specifically, the v8 engine changes regarding the ARM architecture support and this one too. Also the trap handling for FreeBSD I had been ported had been merged recently. Will try to persist in this area, ideally removing all the port patches ... we will see.

dormando, main maintainer of memcached, is more active these last months I believe and had created a new release containing one changes I ve made again for FreeBSD, one about the Dtrace support, the other which was to rewrite the chunk of slabs allocation to be a bit more generic might land a bit later.

AFLplusplus underwent a lot of changes since the last release, mainly by hexcoder and vanhauser-thc. I did some of my own mainly in the multi os support area and few new features too. Following this, I proposed nicely to Gerard Pfeifer, the maintainer of gcc ports in FreeBSD to enable plugin support so the AFL gcc fast mode can be finally built ;-) and he equally nicely accepted it for now only gcc10 but will be back ported to lower versions in the near future.

Revealing the inner power of LLVM

Dear folks,

Here a little important fact in the LLVM history is occurring ... Finally the move from SVN to Git is ongoing ! That sounds very little, but being fully in git rather than using git -> svn conversion tool (or still using svn repo on the side) will be more convenient, I think, as a committer. Appreciated !

Speaking on LLVM, I still try to push in the OpenMP "side of things", here porting simply to OpenBSD, planning DragonflyBSD for next. Hopefully, should be merged after the migration ...

Apart of this, the work been put into libuv had paid off and as a result the 1.33.1 version had been released since few days and being used into NodeJS master branch already, fixing some issues with lesser supported platforms from Haiku (always appreciated supporting this unique OS) to OpenBSD.

At least, I had the opportunity to visit the Facebook's HQ in Dublin, they were presenting their internal (but open sourced) technologies (Folly) and was intrigued why they would not just use the "almighty" Boost ensemble and the standard C++ library. It is mainly for high performance reason , somehow with the incoming C++20 features, it might become a bit less relevant, but their reasoning made sense. The video might be available on Youtube at some point for the curious. "Et voila" :-) for now the radare team is preparing the awaited 4.0.0 branch which is a major step forward, I myself dive into WebKit a little and other little things in the meantime. Until then, I wish you well !

No ... I m not falling in love with Microsoft

but I have to admit they have interesting open source projects :-) True they have great interest supporting FreeBSD, that's a given ; but that is a great benefit regardless I think ... While mimalloc has a well deserved "summer time", I started to look somewhere else in the meantime and stumbled upon snmalloc, which sounded promising especially in multithread context, just looking at the description.

Benchmarking this is always very tricky, always depends on the nature of the software you override memory allocation with, you can always come up with a conclusion while your close neighbour would shake your shoulder and showing you a very contradictory output in his side ...

Nevertheless, I gave a try with several professional and personal projects of mine and what I notice usually is ... snmalloc performs a bit better than hoard but below mimalloc (still remarkable from the rest of the "pack"), with single thread applications. However, snmalloc is above the latter in multi thread context. More generally, my tests displayed they all perform better than the Linux's system allocator in the vast majority of the case. I usually played with hoard but the development is sort of frozen since a while (diehard from the same author is a nice toy to study by the way) or jemalloc before ; but those new players give nice extra choices to look at. As a result of my appreciation, I just pushed a basic openbsd support proposal :-)

This week had been a nice surprise to see American Fuzzy Lop "resurrected" on github in the official google account. Indeed, there was not much activities since the well known 2.52b version. For sure, libFuzzer from LLVM is great but like always, it is better to have multiple screwdrivers around you and I kinda like the sense of humour of this software, if you look carefully while it runs you will understand :-) but the results are very serious, its reputation is well deserved. Now trying to bring improvements ideas for BSD and mac I had since long months ; we will see.

Apart of this, LLVM 9, the first Apache2 licensed version, should occur end of August ... September maybe if more bugs are found. One noticeable thing in FreeBSD's side, for the future LLVM 10, there is a possible interesting fix ongoing from a famous CHERIBSD contributor, hopefully will make it soon-ish and I think it really deserves to be backported to the 9's line. Redis had merged my little FreeBSD support improvement, nodejs as well accepted my tiny OpenBSD build fix.

As American Fuzzy Lop would say "We're done here. Have a nice day !"

Shy sun

Hi folks,

Hopefully you are enjoying the sun, wherever you are ; resting from all high tech things and job possibly :-)
For the rest of us, there are still things ongoing. Little hook here, I wanted to mention the fact Diablo 1 works fully now under OpenBSD, it has been mentioned a couple of times in Twitter and so on, but now the multiplayer works too :-) Congratulations for this ; it must have given lot of joy that's for sure. The next important event in the BSD work would be the vBSDCon the program should appear not long after I write this post ; I hope you will enjoy it hopefully it will recorded.

Following up on the web technologies post, openjdk 11 and 12 ports on FreeBSD; 11 on OpenBSD had been updated, reflecting work from primary battleblow, bsdkurt and a bit myself. As well, I spent most of the time between nodejs (large pages had been accepted and now I try to fix few glitches here and there before next major release), php 7.4 and the future 8 (as I write the 7.4 branch will be frozen pretty soon) fixing little common bugs and improving support for FreeBSD and macOs ; also trying for the first time contributing to the v8 javascript engine to have a sense of their mindset and pace, basically a tiny change for macOs and a more significant one for FreeBSD. Also got to know better the main maintainer of Microsoft mimalloc who is very responsive to say the least, for now two/three changes (large pages for macOs and trying to make it more reliable for Linux) from my part but a lot ongoing for Windows ; it just comes from the daily usage then improvement ideas pop up time to time and I am glad he is open to these when they make sense.
Now time to rebuild the whole FreeBSD current machine, in the meantime I'll continue watching the Artic which is to me a very nice, if not the best, performance from the danish actor. Wish you well ;-)

To be continued ...

Hi folks,

This week, OpenBSD has again provided an interesting update, this time in the graphic driver department. That's quite some work ! For every owner of these components, user of snapshots or current, you re surely giving a try ;-) ... For this post release, I am trying to update the barony package I maintain ... we shall see.

Also, I found few projects worthy to look at ... for instance :

- Speaking of games, PGE Project giving opportunities to create nice games even to terrible game designers as me :-)
- Xonotic here I follow closely a fellow BSD porter for this pretty famous and funny game which works more or less well out of the box on BSD systems in general but it is good to improve the support while decreasing hackeries.
- Every one knows curl but I was pleasantly surprised they were using LLVM lib buzzer (among other things) and pretty cleverly.


These last days, I have been doing little developments lastly for botan, again for Android, to support better older devices (but not that old) regarding auxiliary vectors. Support BSD for the game (N)Blood, as it is based on Doom engine, was pretty straightforward to do ... PHP/swoole, a sort of go(lang) routine equivalent for php, ported for BSD as well ; professionally speaking I m trying to push few changes for python languages existing since last year (but with more than a thousand of pull requests that is a bit normal) the most trivial was merged, I hope the next will make it (ie blake2 module update I mentioned awhile ago) :-) (edit: it has been merged just the following day so all good ;-)). There are still ongoing changes (one of them concerns a little change for PHP and windows specifically ... and more important the capsicum api support).

Wish you well while sneezing due to spring allergy :-P

OpenBSD and post release ... status

Hi folks,

Had been warned by other openbsd folks there is a new MAP_CONCEAL feature (and consumers called malloc_conceal/calloc_conceal (a realloc_conceal is absolutely not necessary, the address will keep the flag internally), I decided to rebuild the whole system despite having done this just last night. This is interesting feature, controlling the informations you want to disclose for debugging. To be more concrete, let's use a very basic C++ application reading an external file containing sensitive information, purposely generating a core dump :

#include <sys/mman.h>

#include <sys/stat.h>

#include <string>

#include <stdio.h>

#include <errno.h>

#include <signal.h>

#include <stdlib.h>

#include <iostream>

int main(int argc, char **argv) {

...

                auto st = fstat(fd, &s);

                auto sz = s.st_size;

                auto flg = MAP_PRIVATE;

#if defined(HIDEIT)

                flg |= MAP_CONCEAL;

#endif

                auto p = ::mmap(nullptr, sz, PROT_READ|PROT_WRITE, flg, fd, 0);

                if (p != MAP_FAILED) {

                        auto str = reinterpret_cast<char *>(p);

                        ::raise(SIGSEGV);

                        ::munmap(p, sz);

...

        return 0;

}

The version without the new flag would disclose the value of the mapped file content as you can see

gdb ./test_without ./test_without.core

...

(gdb) f 1  

#1  0x000006c2da1db473 in main (argc=2, argv=0x7f7ffffc3548) at test_conceal.cc:26

26 ::raise(SIGSEGV);

(gdb) p str

$1 = 0x6c5d77c5000 "My credit card number -- whoooops\n"

...

whereas the version with the new flag ...

gdb ./test_with ./test_with.core

... f 1

#1  0x000006e7271c1484 in main (argc=2, argv=0x7f7ffffcc868) at test_conceal.cc:26

26 ::raise(SIGSEGV);

(gdb) p str

$1 = 0x6e9b7336000 <Address 0x6e9b7336000 out of bounds>

It can be useful that's for sure, we shall see if other operating systems would implement similar feature (edit they do but via madvise) ...

Also if like me you possess a Thinkpad with AMD Ryzen, I would recommend following up this topic from openbsd-misc. The GPU driver had been updated but still the kernel need to be updated accordingly before being able to feel its full potential ...

To finish with OpenBSD, there who is still very active on advertising video games, had updated the list, just to remind there is quite a handful possibility to entertain yourself even on a pretty secure os as this one ;-) His enthusiasm is a nice thing to see !

Day two ...

... "Et voila"!! it was a nice first AsiaBSDCon personally, for this second day I had hard time to choose which talk was the best, especially from the afternoon ... some serious contenders :-)

- Porting go in netbsd arm64 with Maya Rashish ... It is much less easy than it appears, "just" to display the hello world demo, a lot of assembly needed to be done !

- Another path for code quality ? Automated software verification and openbsd with Moritz Buhl, the topic by itself interested me very much even though it seems there is, for now at least, less return than work need to be invested in but still worthwhile, the more code verification means used the better ...

- Improvements to Virtual Machine State Save and Restore / Live migration from Bhyve from Darius Mihai

and Mihai Carabas ... pretty interesting duo for an interesting topic ! The most interesting part to me was the problems brought about clock source in one hand and the migration of memory state in another and which solutions they brought to solve it.

- Finalizing booting requirements for a guest running bhyvearm with Mihai Carabas which to me expanded more into the previous talk, that allowed me to learn more about Arm architecture.

- Finally the "overoptimistic" (means in the end the speaker thought it was already lunch time but we were well behind :-)) but nevertheless interesting talk FreeBSD - Improving block I/O compatibility in bhyve with Sergiu Weisz, again an expansion of the virtualisation topic on FreeBSD focusing on solutions provided for file format supports.

I wish I got more as in the mid day the jet lag caught up and got the best of me :-) but overall I was very pleased with what I heard and saw.

I was able to do little hackery in between talks but I m impressed the level of hackery of other folks around me :-) Time now to enjoy this last day and in the meantime take care and enjoy this end of week end !

Japan's tour

Moshimoshi folks !

As mentioned in previous posts, there is AsiaBSDCon, held in the University of Sciences in Tokyo,  ongoing and already lot of interesting talks went by. 

Similar as last year, there is my personal preferences :

- Adventure in DRMLand with Emmanuel Vadot.

- Intel HAXM/ NetBSD kernel by Kamil Rytarowski.

- bhyve arm64 by Alexandru Elisei.

- powerpc 64 support on FreeBSD by Piotr Kubaj.

So as you can see, almost as the time order presents it ! Also in a personal level I enjoy discovering for the time this country and appreciate the exuberant "eccentric" nature of the city, the vast majority of the population is very smiley and welcoming, not counting the not less surprising meals and incredible stores (especially electronic and similar). That was also the opportunity to expand on my previous talk in Romania last year, so I gave an updated version in the beginning of the afternoon with quite some stress while forgetting some of the points I wanted to mention to be honest :-) but seems some of the attendees appreciated this ...

Tomorrow will be not less interesting I have yet to decide which to attend though, will keep you posted ... While away, some of my previous submissions had been accepted, first radare2/rabin2 will now display the full compiler toolchain used (Elf only for now), Stalker/Xray-16 had accepted my little additional metadata in the debug console, hopefully more to come soon-ish in the meantime I wish you well !

Just in time ...

.. well ... more or less :-)

Indeed, this week, PHP main developers had decided to push (one more time would say some "old timers") a JIT compiler solution, based on DynASM from luajit, which will be present at least for the future 8.x version maybe for 7.4 (as experimental feature). All use cases might not benefit it for sure (and again it is mainly for the x86 architectures anyway) but that might push some devops to consider php to other tasks/contexts than just the dear Web. Here, if you are interested, is the Merge Request still in review when I write for further technical details, I might say that something worthy to look at even though it is "concealed", for now, into the opcache module.

Other than that, these last days I did a sort of indirect contribution for LMMS, more specifically its custom memory allocator named rpmalloc. Just improved a bit the FreeBSD/OpenBSD support (super page support for the former) but alone this project is pretty interesting if, like me, you re used to play with memory management runtime replacements like jemalloc and so on ... To finish I was pleasantly surprised one of my old patches for OpenBSD had been merged (I almost forgot must have been early last year or before :-)) ... just to get the name of the thread ; at least to be equal to NetBSD for example.

Jacket not warm enough ...

... in this end of January, finally the coldness has decided to show up :-) hope you are all fine whatever your plans are. First, sorry for the motivated people willing to go through the last meetup but finally most of the other organisers could not make it. Professional life or little health issues had came in our way in the last moment. We will try to make it happen as soon as we can ...

Since the last time, OpenBSD started, carefully, to import LLVM 7 (in base) while backporting fixes for the code generation. The port version is planned too as you can see here. Also, for the non hardcode followers, OpenBSD default terminal font had been changed/updated (depends how you see it :-)), which frankly quite rejuvenate it, it is definitely more modern looking to my tastes at least.

In a more personal level, I have been doing quite a lot of Android (NDK) development professionally but still had time to contribute to some of the projects I usually overwatch. For instance, php 7.4 will give FreeBSD the possibility to support the "huge" page features for FreeBSD for the opcache module (thus different than the super page support I mentioned a couple of months ago since it resided in the general zend engine memory allocation). I ve had proposed a little update for the network interface lookup functions too which seems to be accepted (basically displaying also if an interface is up or not) but not merged yet.

I will push further new features to radare, one is the possibility to display a binary hashes suite from rabin2 (for now only visible from r2).

capsicum into LLVM ... finally

Hi folks,

In a previous post, I mentioned I worked into capsicum API interception into LLVM's sanitisers. Today, I was finally able to commit the last bits. Thanks to Ed Maste and also Mark Johnston who happily welcomed these with open armed (and pretty easily). As a reminder, the goal is to intercept ,when the capsicum api read/write the rights permission bits, to see if these capsicum data structures are valid.

Another reminder, doom 3 FOSS client is finally now released, it might know be played as much as yquake2/ioquake3 for example in the OpenBSD community but I know some folks who will gladly do ;-)

Speaking of game, there is a new promising one, Spacepunk, from the same who authored Barony. In an early stage for the moment, but there is a good chance it will be appealing in the end (and possible to contribute too), I am pretty confident about this ... I wish more commercial games would follow this business model.

Following-up on porting getentropy/getrandom in the"outside world", gnupg (more exactly its dependencies) might benefit it. My small change had been reviewed, seemingly accepted but yet to be merged.

Do we say register or registry ? I ll look into my super page ...

Hi folks,

More than 4 years now I contribute (seriously) to various FOSS projects and remembering how and why ... I have primarly prioritised contributions to other people/organisation projects rather than my own, especially the ones which I think are useful, meaningful for a whole lot of people. Personal projects are nice to some degree, but I would get bored rapidly and feel too "complacent" if I did that only. But, saying that ... I'm absolutely not willing to convince anyone to agree :-) everyone has its own journey.

So to the point ... these last days I have been focusing on porting or more precisely trying to equal features with Linux. For example now redis can dump backtrace and its registers on BSD and ported to FreeBSD RedisSearch (was easy enough I admit), php might start to support huge pages functionality (from Zend engine POV) already merged, I m trying at the moment to port it in the opcache module as well. radare as usual right after its release the usual contributors started to fix various places, now the FreeBSD package had been updated today. OpenBSD ought to follow soon-ish.

Post (about) meetup

Hi folks,

Finally we made it happen and was the opportunity to get to know Tom Smyth, very well versed into network, ISP and also a big fan, in a personal and business level, of OpenBSD. I quickly saw him in EuroBSDCon but in the meetup I could witness his taste of convincing how OpenBSD can serve his security priorities, how well it is to fund worthy open source developers for the greater good and so on ... Was a great discussion to listen to. Of course seeing the other usual folks was great too ;-) which lead to think we should go on on "reviving" our group so we will see each other again in December (hopefully), the topics will be make public soon-ish but we will basically expand on topics we did not have time to reach.

Apart of this, my little redis changes had been merged, I may have further plans while also thinking of redis "mindset" i.e "not too much platform "boilerplate" please..." ; to summarise. radare2, again, merged some of my fixes and especially the DragonflyBSD build, thankfully despite being different than FreeBSD still have, to some extent, a similar-ish API.

At last php has moved towards the 7.3.0 release, at least the RC5 had been sorted out in due time so not so long after we should be "back in business" about taking care of new features for future versions. Definitely, the opcache.preload feature/directive will be one of the main point of attention (in my opinion). Finally, my merge request for openssl refactoring had been accepted.

Time changes ... or changes time

Hi folks,

Reading this past PR when we can see finally Kamil is moving on. Whoever takes over, there is quite a legacy ! Without him, the NetBSD support into LLVM would not be where it is now that's for sure (not counting his contributions to NetBSD itself). Thanks for the numerous reviews since last year ;-)

Also quick reminder, on Thursday our little noisy discussion about ZFS in Dublin 2 ... Seemingly there is an additional talk/introduction about OpenBSD ... nice.

These days, it has been more radare than anything else, to the project itself also trying to update BSD packages (the OpenBSD's version should be merged today or so ... while I write, the two maintainers agreed it was really time to do so, it was very old version).

But I also had my little take into memcached to improve a bit the FreeBSD support, hopefully some will land into the next release.

After that, I am waiting patiently php master "unlocks" (maybe after RC5) so new features will get merged.

BSD can be narcissistic too

Hi folks,

Couple of updates since the last time, I have began to port the radare's self plugin for the BSD, starting with FreeBSD then next should be OpenBSD and finishing with our dear friend NetBSD so time for them to have some introspection if you will ;-) Will try to make it available for the next major release at least.

Also tried a bit redis recently, fixing FreeBSD build. similarly a little build fix for clangd (aka clang Daemon) a settings server for editors. It is just a matter of making people aware of BSD specificities, as usual, and  generally they are pretty open to this idea ; in the long run it ends up positively in the vast majority of the cases, at least in my personal experience ...

Going through, as mentioned in a previous post, the next FreeBSD quarterly report, one of them states than some FreeBSD and Intel people are organising a sort of team to be able to provide a better Intel support at large and security matters as well. Seems the Spectre/Meltdown issues brought a positive outcomes after all.

Let's meet up again

Finally it s getting concrete :-) after months of silence there is the next meetup occurring the 8th. We ll try to cover as much as possible, ZFS alone is quite a topic by itself ... but as always it will be a small group but dynamic ... more an exchange of knowledge ... Everyone knows a little something :-) Also the opportunity to cover conferences people might not have attended. As you can see the place is quite nice and largely sufficient to handle a reasonable group so hopefully we will see you over there.

Speaking of Dublin, Monday is public holiday, will be the opportunity to move on on some things which went stale a little. My previous OpenBSD gear had been converted into a real FreeBSD development one, largely sufficient for even LLVM with real parallel build support. By the way, speaking of AMD Ryzen, Larkin (at least) is looking for having a hand on this pretty recent hardware to fix proper hibernate/resume ... Just in case.

In the previous post were mentioned the infamous xorg issue which now have a fix for current so soon should follow a fix via syspatch for the 6.4 release ...

Better later than never

Hi folks,

Here is the first (indirect) OpenBSD ports contribution post 6.4 release, so finally pypy changes had been merged. Also another late merge this time for LLVM 7.0.1 as I promised in the last EuroBSDCon to the attendees my fix for X-ray tooling had been backported.

Speaking of conferences, there is the last MeetBSD which took place in California and seemingly had been a great success reading some people reports :-) the Mariusz Zaborski's talk was surely a great one but also two from last EuroBSDCon like Kamil's talk or "What (not) to Monitor" from Andrew Fengler. Seemingly the next "important" BSD event will be the AsiaBSDCon one. We shall see ;-)

Apart of this, few small fixes here and there, also awaiting new sdb release to fix a longtime issue. At last, Haproxy 1.9 release should occur soon-ish. Seems a very much expected one ;-)

Under the radare

Hi folks,

I think now it is safe for the radare FreeBSD port maintainer to update the version as the 3.0.1 release is out.

At the moment, focus are into LLVM 8 for OpenBSD and especially FreeBSD.
For the OpenBSD build, disabling retguard and stack protector features seems to be used, to avoid large creation of their ELF sections and not be able to launch the executables. For FreeBSD, time to switch to the new 13th branch :-)

These days, I did a lot of golang professionally and discovered golangci-lint existence from one Dublin meetups contacts and was able to compare with more known gometalinter. The former might be appreciated from mid-size code due to its faster performance. Also able to highlight the "faulty" code. I recommend.

Countdown ...

Hi again,

Here you can see how many days left before OpenBSD decides to bump the versioning here ... Noted also that even though it is disabled wisely for now because of the tree lock, lldb had been imported. On OpenBSD it can debug a simple core dump, but can't attach to a process. But that is already a lot (and there is gdb from the ports too) ! so awaiting the tree unlock in the following weeks ... So great work from the small OpenBSD source team (as usual, but they never disappoint).

Noted as well BSDMag has been inactive for couple of months (at least in appearance), no new issue since the last time, hope all is fine over there in Poland ...

In a personal level not too much since, only I did a very little contribution to lugaru (was the opportunity to see I miss C++ some days :-)) also not visible yet but takes a bit into NetBSD world (I mean more than usual) as I do not necessarily like to limit my world and like to discover strengths from other perspectives.

From LLVM point of view, after Esan for FreeBSD is completely merged (frontend bit), I ll have a bit of Darwin work (possibly), we ll see.