TRANSFORM 2022

Last week the Software Underground hosted the latest edition of TRANSFORM, its annual festival of digital subsurface stuff. This year we had a packed schedule, including:

• Folks sprinting on various software projects at the weekend.

• 13 live hands-on tutorials on a wide range of topics.

• Birds of a feather meet-ups to discuss the future of scientific virtual meetings.

• Eight lightning talks on topics from sealice to web apps to scholarly communication.

• The Annual General Meeting of the Software Underground organization.

All this was stitched together by the Slack chat, as usual. You can catch up on what you missed in the #t22-general channel. Our Slack is free to join and open to anyone.

---

The tutorials

Here’s a list of the tutorials, which were very high quality this year. All of them were 90 minutes to 2 hours long, and all are supported by fully open-source software and open-access materials. How awesome is that?

• Getting started in Python, by Robert Leckenby (Agile).

• GSTools, a toolbox for geostatistical modeling in Python, by Sebastian Müller (UFZ, Leipzig).

• What is Markov chain Monte Carlo? By Miguel de la Varga (Terranigma).

• Mandyoc, a finite element simulator for the mantle, by Agustina Pesce et al.

• Machine learning models for geoscience, by Tom Ostersen (Datarock).

• From Zero to Devito (geophysical modeling & inversion), by Rhodri Nelson (Devito).

• Publishing your first Python package, by Matt Hall (Agile).

• PyGIMLi (geophysical modeling & inversion), by Florian Wagner et al. (RWTH).

• OccamyPy, an object-oriented optimization framework for large-scale inverse problems, by Francesco Picetti & Ettore Biondi (Stanford).

• A geophysical tour of mid-ocean ridges, by Leonardo Uieda (U of Liverpool).

• Self-supervised noise suppression, by Claire Birnie & Sixiu Liu (KAUST).

• Julia for geoscience, by Francis Yin (Georgia Tech).

• PyLops (geophysical modeling & inversion), by Matteo Ravasi et al. (KAUST).

If any of those sound good to you, and I hope they all do, check them out in this YouTube Playlist.

---

The lightning talks

As always, the lightning talks were a lot of fun. Check out the Description in this video for the full list of chapters:

---

That’s it for 2022. Hope to see you at TRANSFORM next year!

News from the Underground

In the seemingly random content factory that is the Software Underground community, there has been a heavy sampling this past week around openness. What is open source? What is open data? What are good practices? and, How does this affect me? Here’s a collection of those conversations and more.

The Geothermal Hackathon — happened last week immediately following the World Geothermal Congress Geoscience virtual event. You can read about the things that people built leveraging open data sets and be sure to connect with these creators in the #geothermal channel.

The complicated world of open source — Spilling over from an SPE workshop on Open Software, is perhaps the longest thread in Swung history about what actually constitutes open source software, why it can be so confusing, and what the implications are for scientists and technologists. Matt followed up with a number of suggestions how technical societies can support openness, and also created a poll to measure the degree of confusion around open source. Conclusion: it depends.

A checklist for open scientific software — Yes, open source is complicated, especially for newcomers, so it seems fit for relatively straightforward tools to guide behaviours. Matt shared a so-called best-practice checklist for open scientific software, which quickly underwent a handful of revisions after some supportive feedback. It is meant to be more than just tick marks on a piece of paper but that it can be a vehicle for delivering behavioural change.

Big Borehole Dig – Steph shared a cool project launched by the British Geologic survey welcoming scientists and citizen scientists alike to digitize their vast collection of historic logs into a standard digital format. It’s the ‘ol PDF to actually-digital transformation challenge and this one is a tall order. But just imagine the data science possibilities from 1.4 M boreholes!

Tools and tactics – people are getting help on to tricky technical questions in the #python channel on a variety of topics including: dealing with very large tabular data with Vaex, how to constrain solutions to non-linear problems with scipy.optimize, and fixing missing data values in rasters with rasterio.

Vedo – in the visualization awesomness category, the winner goes to a post that Dieter made in the #viz channel about the Vedo project, whose gallery will incite all the feels of a kid in candy store for those working in 3D. Notably, the first tile in the gallery is a demo geo-model shared by Richard Scott. Check out his scene here before you get on with the rest of your day.

Vedo – a python module for scientific analysis and visualization of 3D objects.

News from the Underground

The reverberations from TRANSFORM 2021 have started to dampen, so we’re back with our weekly round-ups.

Badges of honour — Did you know that you can slap a Software Underground badge on your projects and webpages? These badges aren’t just for showing off either. They can links to other places, for instance to your project’s channel in the Swung Slack, if it has one.

Geothermal Hackathon 2021 — is taking place at the end of next week with much activity to be going on in the #geothermal-hack-2021 channel. Here are some of the proposed projects. Even if you aren’t registered in the hackathon you can still follow the action.

Superstar with LiDAR — Are you dealing with .las files, but not of the wireline kind? Then this discussion will serve you up a platter of resources and advice for working with point-cloud data sets.

Feature combo pipelines — If, like Brendon, you’re doing some machine learning and you find yourself building a boatload of models all with different combinations of inputs and pre-processing steps, you’re life is going to be much easier if you have a way to keep track of these different permutations so you can apply it all again on other data.

Need a license for work — Pythonistas using Anaconda might be shocked to find out that it’s no longer free to use. Sort of. It depends on who you work for and the kind of work you are doing. This thread discusses the issues and reveals a shared frustration in comprehending the terms and conditions. There are also some thoughtful perspectives on supporting the free (or nearly free) software that many organizations become increasingly dependant on.

Tools for seeing — Geoscience very often relies on making graphics and visualization. This post pointing to the Python-only compilation called PyViz.org, shows you the collection of tools that are out there. It looks to be a comprehensive resource, particularly because the number of tools in the tool shed can seem so daunting.

Transform 2021 rolled out

On Friday we wrapped up the 2021 edition of TRANSFORM, the Software Underground’s annual virtual conference. We stuffed a hackathon, 21 tutorials, 20 lightning talks, and an annual general meeting into a week-long celebration of open subsurface code and data.

Many thanks to our sponsors — especially Studio X in Austin, Texas — and to all of the participants who donated to the conference this year. As a pay-what-you-like event, we depend on generosity to fund the things we do. In return, we are trying to bring some new superpowers to the community. So far, so good.

Just like last year, the event kicked off with a hackathon, which again lasted all week. The event saw 9 projects being worked on. this number included long-lived projects like Subsurface, GemGIS, SEGY-SAK and Striplogm as well as new projects looking at seismic footprint removal and well correlation, among other things. We’ll tell you all about these projects in the coming days.

The heart of the conference week itself was the 21 amazing tutorials — 33% more than we had in 2020! The instructors and presenters this year participated from all over the world:

The 29 instructors and 20 presenters this year came from all over the world.

We’re so fortunate to have scientists in our community that not only spend hours (years!) writing open source software for others to use, but then will also contribute open content to help others use it. Many, many thanks to our team of instructors:

Day 1: Ashley Russell (NOR), Claire Veillard (NOR), Florian Wagner et al (DEU), Anne Estoppey (NOR), Miguel de la Varga (DEU), Diana Acero-Allard (USA), Maria Cecilia Bravo (NOR), Michael Pyrcz (USA), Thomas Martin (USA).

Day 2: Tony Hallam (GBR), Lachlan Grose & Mark Jessop (AUS), Olawale Ibrahim (NGA), Øystein Klemetsdal (NOR), Nathaniel Jones (USA), Bane Sullivan (USA), Seogi Kang (USA).

Day 3: Jørgen Kvalsvik (NOR), Graeme Mackenzie (NOR), Edward Caunt (GBR), Steve Purves (ESP), Matteo Ravasi (SAU), Dewey Dunnington (CAN), Santi Soler (ARG), Irene Wallis (NZL), Katie McLean (NZL).

It wasn’t all Python either — one tutorial used R, another MATLAB/Octave — and we even had a tutorial in Spanish this year. We also had participants from well outside the usual petroleum-rich industries: geothermal, mining, hydrology, and technology were all represented. I hope we continue to see increased diversity in languages, industries, location, and in all dimensions.

The best thing of all, especially if you weren’t able to take part last week? Everything is on YouTube, so check it out, follow along, and learn some new tricks.

One of the really remarkable things this year was the prevalence of tutorials and hackathon projects that combined multiple software projects from the open subsurface stack (which was the goal of TRANSFORM all along). Several events combined two or more of GemPy, Devito, PyVista, Subsurface, segyio/SEGYSAK, or lasio/welly/striplog. It’s exciting and encouraging to see, and signals a new level of maturity among these tools. The future for the open subsurface stack looks bright.

Thank you to the organizing team — Dieter Werthmuller, Ashley Russell, Irene Wallis, Brendon Hall, Rob Leckenby and Sofiyah Mokhtar — for helping pull everything together.

And thank you to everyone that participated in TRANSFORM 2021. I hope you learned something new, and met someone interesting. Please tell your friends and come back next year!

News from the Underground

Here’s all the news from the Underground this week…

Have you signed up for TRANSFORM 2021 yet? Because it kicks off with the hackathon next Friday 16 April. The hackathon projects are starting to take shape and there is going to be a special Rendezvous session on next Wednesday 14 April at 16.00 UTC to discuss project ideas. Come along and let the project ideas wash over you and vie for your attention.

What to learn first? — Matt created an infographic entitled, Which language should I learn first? with some opinions about how to get started in programming for science, business, and fun. There’s more context in his blog post and this thread contains some good perspectives from others. There’s quite a bit of chat on Twitter about it too.

Colour science for colouring science — Alessandro’s posted in the #viz channel an excerpt from one of Edward Tufte’s books, appealing to those particular colour orderings that “have a natural visual hierarchy”. Check out the thread for some thoughtful considerations concerning the state of the art for colouring data.

Image modified from fabriocrameri.ch.

Singing with the birds — Jesse shared an article in #random with the opening line, “I had a dream one night where a blackbird was talking in human language”. The article then ventures into an exploration of converting human speech into birdsong. If you’re into signal processing, acoustics, and ornithology, you’re going to think this is really cool.

As always, if you want to highlight anything in these weekly posts, message @Evan on Swung.