Transform 2021 rolled out

On Friday we wrapped up the 2021 edition of TRANSFORM, the Software Underground’s annual virtual conference. We stuffed a hackathon, 21 tutorials, 20 lightning talks, and an annual general meeting into a week-long celebration of open subsurface code and data.

Many thanks to our sponsors — especially Studio X in Austin, Texas — and to all of the participants who donated to the conference this year. As a pay-what-you-like event, we depend on generosity to fund the things we do. In return, we are trying to bring some new superpowers to the community. So far, so good.

Just like last year, the event kicked off with a hackathon, which again lasted all week. The event saw 9 projects being worked on. this number included long-lived projects like Subsurface, GemGIS, SEGY-SAK and Striplogm as well as new projects looking at seismic footprint removal and well correlation, among other things. We’ll tell you all about these projects in the coming days.

The heart of the conference week itself was the 21 amazing tutorials — 33% more than we had in 2020! The instructors and presenters this year participated from all over the world:

The 29 instructors and 20 presenters this year came from all over the world.

We’re so fortunate to have scientists in our community that not only spend hours (years!) writing open source software for others to use, but then will also contribute open content to help others use it. Many, many thanks to our team of instructors:

Day 1: Ashley Russell (NOR), Claire Veillard (NOR), Florian Wagner et al (DEU), Anne Estoppey (NOR), Miguel de la Varga (DEU), Diana Acero-Allard (USA), Maria Cecilia Bravo (NOR), Michael Pyrcz (USA), Thomas Martin (USA).

Day 2: Tony Hallam (GBR), Lachlan Grose & Mark Jessop (AUS), Olawale Ibrahim (NGA), Øystein Klemetsdal (NOR), Nathaniel Jones (USA), Bane Sullivan (USA), Seogi Kang (USA).

Day 3: Jørgen Kvalsvik (NOR), Graeme Mackenzie (NOR), Edward Caunt (GBR), Steve Purves (ESP), Matteo Ravasi (SAU), Dewey Dunnington (CAN), Santi Soler (ARG), Irene Wallis (NZL), Katie McLean (NZL).

It wasn’t all Python either — one tutorial used R, another MATLAB/Octave — and we even had a tutorial in Spanish this year. We also had participants from well outside the usual petroleum-rich industries: geothermal, mining, hydrology, and technology were all represented. I hope we continue to see increased diversity in languages, industries, location, and in all dimensions.

The best thing of all, especially if you weren’t able to take part last week? Everything is on YouTube, so check it out, follow along, and learn some new tricks.

One of the really remarkable things this year was the prevalence of tutorials and hackathon projects that combined multiple software projects from the open subsurface stack (which was the goal of TRANSFORM all along). Several events combined two or more of GemPy, Devito, PyVista, Subsurface, segyio/SEGYSAK, or lasio/welly/striplog. It’s exciting and encouraging to see, and signals a new level of maturity among these tools. The future for the open subsurface stack looks bright.

Thank you to the organizing team — Dieter Werthmuller, Ashley Russell, Irene Wallis, Brendon Hall, Rob Leckenby and Sofiyah Mokhtar — for helping pull everything together.

And thank you to everyone that participated in TRANSFORM 2021. I hope you learned something new, and met someone interesting. Please tell your friends and come back next year!

News from the Underground

Things were pinging away in the main channels this week, with a few posts garnering loads of enthusiasm and important chatter.

The weirder the better — In the #viz channel, Nathan announced the latest Xeek challenge, Keep Geoscience Weird. This competition is all about new and novel ways of visualizing subsurface data. It’s a very open-ended competition — you can use any data that you like, as long as it’s related to the subsurface. I highly recommend reading the thread concerning the Terms & Conditions and motivations for engagement.

Sand vs shale — The longest thread of the week (150+ messages!) was in the #sedimentology channel. A simple question about computing running averages led to uncovering a bug in striplog, some plots (right), a hackathon project, and some weirdness in how Microsoft Excel exports CSV files.

Hello OSDU, how do you do? — The Open Subsurface Data Universe (OSDU) Forum is a consortium of ~200 members and is rolling out the first production release of its platform this week. For a bit of history on how the OSDU came to be, who’s involved, and where it’s headed, this thread might be a good summary.

But seriously. Who’s with me? — Has anyone ever given any serious consideration to prospecting and mining asteroids or other planetary bodies for resources? Turns out they have.

Teeny weeny ancient plant bits — Kieran threw some Carboniferous kerogen samples into the SEM and shared these gorgeous images. The discussion drifted naturally to “should I put 300 of these images online somewhere for others to look at and play with?”

How to get lost — Kleo asked for recommendations for introductory Python exercises, and got a load of responses. A few folks pointed to last year’s TRANSFORM tutorials. In another channel, Heba got a somewhat philosophical perspective from Hallgrim on the pursuit of programming: “[it’s] all about getting lost, banging your head against the wall, and the extreme pleasure of breaking through the wall. You are already lost, you’ve already found this community, so you have a great talent for what is ahead of you…”.

What was your favourite thread of the week?

News from the Underground

Here’s a selection of newsworthy things over the past week.

Nice to see you. Can you help me? — New to the Slack? Here’s a friendly post that if you do in fact introduce yourself like Elvis did stating some of your interests and ambitions, the first connections may come from other people reaching out to you with their technical questions. How cool is that?

Real-time coding collaboration — Ever wondered if there are platforms where you and your chums could collaborate in real-time on your code? Check out the handful of options that came up in this post.

Rendezvous for the hackathon — The next Rendezvous is on Wednesday (not the usual Friday), 7 April at 4 pm UTC. This one is a bit different because it will be a group chat about the projects happening at the upcoming TRANSFORM hackathon. So if you have something in mind, this is your chance to pitch it. Better yet, start a discussion in Slack right now!

Kudos and logos and merch — Doing a shoutout to Software Underground in some graphical format such as a presentation, video, or poster? Make sure your acknowledgements look crisp with the official logos. And while representing, colour coordinate your video calls with a coffee mug, T-shirt, or stickers from the shop.

Shear advisory — A bit of rock physics in the #machinelearning channel as Mads asked for help predicting a shear sonic log. This is a classic seismic geophysics workflow because shear sonic is seldom acquired but allows rock physicists to get into elastic properties.

Tech for online meetings

In pulling TRANSFORM 2020 together, the organizers (Filippo Broggini, Brendon Hall, Rob LEckenby, Victoria-Lou Devezes, and me) looked at a lot of technology options. I thought I’d share the stuff we used here, in case you’re wondering what you need for a full-on virtual conference. But first, I’ll try to give a high-level description of the sorts of events we hosted at the conference.

---

We had three main types of session to support:

Tutorials

We hosted 14 three-hour tutorials, which mostly involved one or two instructors and 10 to 150 learners. Behind the scenes, there was one ‘livestream host’, who was essentially like a TV director.

The instructors were in a Zoom meeting with the host, sharing screens with them. The host captured their Zoom meeting with OBS Studio, and streamed it to YouTube Live. All of the participants watched via YouTube, and the live stream is also preserved there.

Lightning talks

There were two lightning talk sessions, each two hours long. The set-up was similar to the tutorials, except that there were more presenters — 12 per session. They came into the Zoom just before their allotted time and left right after, so there were never more than 3 or 4 people in the Zoom at once.

Unsessions

These were not streamed to YouTube, so we were able to host everyone in one big Zoom meeting. (I had a Large Meeting add-on ‘just in case’ but we didn’t need it in the end; the most we had was 97 people.) This made the session susceptible to Zoom-bombing, so we were careful about where we shared the link, and we used a waiting room — with Victoria ‘on the door’ to let people in promptly.

The unsession format, which I’ll write about another time, relied heavily on Zoom’s breakout rooms. We also used GroupMap a lot — it’s a great tool for capturing ideas from a group of people.

---

Here are the tools we used:

• Slack — the heart and mind of the Software Underground already lives in our Slack workspace. To make the multiple conversations more manageable, we made channels for all of the sessions and hackathon projects, and this worked well. During the week, 700 active members exchanged about 19,000 messages, about 50% of which were in private chat.

• Zoom — the Covid-famous video conferencing tool. It worked well for us for the host–presenter meetings in the tutorials and lightning talks, and as the main room for the unsessions and hackathon presentations. We were very worried about Zoom-bombing, and were perhaps over-cautious (some legit people found it hard to enter sessions).

• YouTube — for the tutorials and lightning talks, we streamed the host–presenter Zoom to YouTube for participants to consume. This has a few big advantages: it eliminates the Zoom-bombing risk, participants can pause and rewind the live stream, and videos go straight to YouTube afterwards. And it’s free!

• OBS Studio — a fantastic open-source tool that lets you combine images, video feeds, and audio sources into a single stream, which you can send to YouTube (or Twitch or any other streaming service). This is how we streamed the Zoom sessions. It does have a learning curve though, and certainly requires an off-screen ‘director’ to manage it all — and several practice sessions to get the workflow down.

• GitHub — is indispensable for code-sharing and source control. I think all of the hackathon projects hosted their repos on GitHub. The tool is not intuitive for new programmers though, and wrangling git and GitHub is one of the most requested help topics in our hackathons and courses.

• GroupMap — a wonderful tool for collaborative brainstorming. It definitely needs a couple of hours to get the hang of what it can do, but for me this was the standout discovery of the event. Its best feature is that you can set up a workflow, like Survey > Brainstorm > Vote > Results and then guide the group through the stages, live.

• Sched and Eventbrite — for event registration. Both of these tools have their high-points — Sched is really nice for building the event schedule — but the registration process for the event was a bit of a hairball and while these tools are supposed to play nicely together, I never felt comfortable with either of them.

• Printful — a T-shirt (and other merch) vendor. The advantages are that they print ‘direct to garment’ (i.e. there’s no setup, they just print a shirt when you order one), they take care of fulfilment, and their system works seamlessly (sort of) with Squarespace, our website host. But the system is not that easy to use and I’m thinking of switching to Teespring.

• We also used a bit of hardware. Microphones were hard to find during the Covid-19 crisis, but we sent Blue Yeti Nano or Blue Snowball Ice USB mics to our instructors. The Yeti Nano is especially nice, with two pickup patterns and a hardware mute button.

• Other… The organizers and the participants used other tools during the conference, including: Mentimeter, Google Maps, Miro, and HackMD, plus of course Twitter and other social media.

---

I don’t think I appreciated it before the event — looking back, it seems obvious — but technology is completely intertwined with an online-first event. Sometimes the limitations of the tool force you to adapt; other times, the tool can enable new things you hadn’t thought of. It’s going to be interesting watching these remote collaboration tools evolve over the coming months and years. And even more interesting thinking about what kinds of meetings we can have with them!

If you have a favourite collaboration tool I haven’t mentioned here, or you’ve been in a meeting that did something different, I’d love to hear about it.

The TRANSFORM 2020 hackathon, Part 2

This is part 2 of a pair of posts about the TRANSFORM 2020 Hackathon. Read part 1 here.

Welcome back to the TRANSFORM 2020 Hackathon project round-up!

Last time we shared six projects; the remaining projects are listed below. As before, to find out more about a project, get in touch with its champion, or visit its channel, in the Software Underground Slack. The channels all start with #t20- so, for example, the GemGIS channel is #t20-gemgis.

---

Seismic processing

Alan Richardson set himself the ambitious task of wrapping CPSeis, an open source seismic processing system, in Python. However, this proved fiddly, so he set himself the even more ambitious task — albeit one with fewer dependencies — of writing a seismic processing system in Python. Punchline: he’s building it with TorchScript. It’s going to be epic. Repo.

---

Covid-19

Artash Nath, a talented 8th grader from Toronto, led a project to observe the impact of COVID-19 lockdown restricitons on the traffic movement in several Canadian cities. Specifically, Artash’s goal was to measure the change in the seismic vibrations in major cities before, during, and after their lockdown period. Repo.

---

GemGIS

Alexander Juestel and Arthur Endlein attempted to meld the worlds of GIS and GemPy. Their goal was to build a 3D geological model from a geological map (a shapefile) and a DEM (a raster). The process was fully interactive, so they could digitize a cross-section right in a notebook. Repo.

---

Geothermal field

Alexis Lamparski and Elisa Heim, along with a few others, analyzed and visualized seasonal upper soil temperatures over a geothermal field. Their goal was to find a correlation with fluid temperature differences inside the horizontal pipes connecting borehole heat exchangers to the heat pump. Repo.

---

Litho-boundary from gamma

Jared Armstrong, after reading a recent paper about generating lithological boundaries from gamma using wavelet tessellations, Jared decided to stay up late and join from Down Under. Jared joined forces with Leo C and Martin Bentley and together they were able to create promising initial results. Repo.

---

Devito

Gerard Gorman and Fabio Luporini, together with other project developers, took advantage of the hackathon to improve the documentation and to start tackling issues related to HPC and distributed computing for the Devito geophysical modeling project.

---

Space rocks

Hunter Danque took a shot at the Moon… using the lunar GPR data from China's Yutu 2 rover. Hunter started working towards his goal to load the data to eventually extract geotechnical properties for future lunar site investigations. Repo.

---

This amazing event was the first big virtual hackathon we’ve taken on. With at least 60 people collaborating on 13 projects, it was undoubtedly a success, but we learned a few things too. It was less accessible to new programmers than our live events usually are. And it was hard to even begin to approximate some of the social aspects of a live hackathon.

One thing is certain: we will be doing this again — and you’re invited. See you then!

---

Many thanks to our sponsors for their generous support of this event