AVS 66 Session RA+AS+BI-WeA: Addressing Reproducibility Challenges using Multi-Technique Approaches

An increasing number of studies, surveys and editorials highlight experimental and computational reproducibility and replication issues that frequently appear in most areas of modern science. In a 2018 AVS conducted survey, 66% of those responding identified reproducibility as a significant issue. There are multiple and complex causes of what some have called a “reproducibility crisis,” which can impact materials, interface/(bio)interphase, vacuum and others sciences of importance to AVS members. Reproducibility challenges are not new, but now appear in both old and new forms requiring innovative solutions. Drivers influencing reproducibility problems include the increasingly multi-discipline, multi-method nature of much advanced science, increased complexity of the problems and systems being addressed, and the large amounts and multiple types of experimental and computational data being collected and analyzed in many studies. Such issues challenge experimental teams and the review process. Systematic and sustained efforts are needed to address the causes of reproducibility problems that can hinder the rate of scientific progress and lower public and political regard for science. The Focus topic New Challenges to Reproducible Data and Analysis aims to raise awareness of the challenges, examine the causes, impacts of reproducibility problems and explore approaches that can help address both the newer and older generation of reproducibility challenges. Some problems and solutions are easy to identify, even if not readily implemented. Other drivers and causes are less obvious and therefore harder to address. This talk will introduce the focus topic sessions, review key literature on the topic of reproducibility and summarize how the presentations fit together as a way to address reproducibility challenges.

Repeatability and reproducibility in surface analysis in the semiconductor industry are key to for supporting efficient process development as well as High Volume Manufacturing (HVM). As two examples, long term repeatability is critically important when comparing to historical data, while reproducibility is required to support technology transfers when HVM of specific devices is to be carried out at multiple sites. This however introduced a number of unique challenges for running a characterization facility.

In this presentation we will present a number of examples that can result in reproducibility issues. Particular focus will be in the areas of X-ray Photoelectron Spectroscopy (XPS) Secondary Ion Mass Spectrometry (SIMS). The first and foremost causes of repeatability and reproducibility arise from instrumental variation. A second important source arises from samples variability. We will show that assessing long-term instrumental stability is potentially hindered by long term variation of samples characteristics. We will also show that an understanding of the characterization techniques is paramount to understanding such issues.

Next to the “pure” technical causes of repeatability and reproducibility, is the human factor. This involve for instance decision making in data treatment during for example, fitting procedures, statistical treatments, etc. This will be illustrated using practical examples. And with present day characterization depending more heavily on computational support/ commercial software, potential detriments to characterization repeatability will again be made evident. Finally, we will show through round-robin results, that combining all the above factors, widely varying results can be obtained on the same samples.

Our publications should serve as guides to repeat our experiments/analyses and reproduce the results; however, quite often we may find ourselves not able to do so. Over the past few years, there have been many papers and editorials that have shown that issues associated with Repeatability, Reproducibility, and Replicability impact almost all areas of science, and in an AVS-conducted survey, 65% of those responding indicated that they have seen or experienced significant reproducibility issues when they have sought to recreate experiments from the literature. It is clear that the increasing demands of complex research requiring use of multiple experimental and computational research methods is a central theme. The challenge in the Biointerface Science community is compounded by the inherent variability of biology. Working at the interface between the physical and life sciences, it is often difficult for us to have in-depth knowledge of the idiosyncrasies of the many techniques we use and we need to be aware of for our data interpretation. This talk will discuss approaches we are taking to tackle this issue within the Biointerface Science Community and the journal Biointerphases. We will look at how we can champion best practices, sharing our knowledge across our community, and seek to support researchers who are new to the field or want to explore new techniques to avoid the pitfalls and better understand both the opportunities and limitations of the techniques, methods, and approaches used in our multidisciplinary community.

Sally L McArthur, Editor Biointerphases

Engineered nanoparticles add high value to commercial products and have the potential to improve our quality of lives and boost prosperity. For example, they provide radical new approaches to cancer drug delivery, biosensing, medical imaging and catalysis. However, the effective implementation of these materials relies on the ability to measure and control their properties, such as their surface chemical identity, size and concentration. There are significant challenges in the analysis of nanomaterials due to, among other factors, the interdisciplinary nature of the field and the lack of adequate reference materials to calibrate analytical tools. The use of complementary tools provides opportunities for (1) deepening the quantitative understanding of these systems and, importantly, (2) a route to method validation. I will provide examples from our work on both these cases.

(1) We use a combination of methods to analyse nanoparticles which are employed in liquid media (in-situ) using techniques such as analytical centrifugation and dynamic light scattering and ex-situ with X-ray photoelectron spectroscopy (XPS). Sound sample preparation protocols are critical for meaningful and comparable measurements. This is especially important when using complementary methods for the analysis of the same samples. I will discuss our experience in the analysis of protein coated gold nanoparticles and polymeric core/shell nanoparticles and show how multimodal analysis is critical to the full understanding of the system.

(2) The lack of certified reference materials for nanoparticle number concentration has hindered the validation of laboratory methods, which resulted in a general distrust in commercially available instrumentation. We have led a collaborative effort to develop accurate methods based on small angle X-ray scattering (SAXS) and single particle inductively coupled plasma mass spectrometry (spICPMS) for the measurement of colloidal number concentration. We have then used these methods to assess and validate a range of laboratory methods. I will discuss the result of this work for both ideal and agglomerated nanoparticles and present the outcomes of a large VAMAS interlaboratory study which assessed four methods for the measurement of colloidal concentration.

Due to the worlds ever increasing energy needs, renewable sources, higher efficiency and energy storage have become important research areas. Therefore, full analysis of the materials used in such applications can add to our understanding of these emerging technologies. In many cases this will mean using several techniques on a single sample.

The chemical composition of the materials found in batteries or fuel cells play a huge part in the desired properties. An example of this is in the inclusion of Sr²⁺ in the A sites of lanthanum manganite. This increases the electronic conductivity of cathode material via the addition of electronic holes to the perovskite structure. XPS allows us to quantify the chemistry of the material and use that data to further improve its properties.

Chemical analysis of the surface of the material by XPS also allows us to identify diffusion or segregation effects that can occur once a battery material has been cycled. Once a build-up of surface material on an electrode becomes too thick ions cannot pass between them, preventing charging of a cell. Using XPS with other techniques like ISS allows us to characterise the surface material (~ top 10 nm) and the surface monolayer.

Here we discuss a LaSrFeCoO perovskite sample typically found in fuel cell and battery cell electrode materials. XPS is used to quantify the composition of the material and identify the La chemical bonding state to find the sample contains La₂O₃ bonding. ISS is also utilised to show no significant Fe and Co at the top surface of the sample. Comparing this to the XPS data taken from the top 10 nm of the shows signs of Fe and Co depletion at the surface.

The perovskite materials are also found in solar cell components. These materials are often used due to their high efficiencies but also because the material band gap is tuneable therefore allowing us to optimise the material composition. Using a technique like REELS combined with XPS can enable us to measure the band gap of the material to reveal the efficiency as well as identify the composition. In this case the band gap of the sample was calculated at 6.3 eV using REELS.

We will also be discussing the use of coincident XPS/Raman to investigate the bulk and surface characteristics of the LaSrFeCoO sample without exposing it to atmosphere between analysis.

A ToF-SIMS / AFM methodology that combine the topographical information with the chemical composition has been established [1]. It was used to achieve a topography-corrected 3D ToF-SIMS data set and maps of local sputter rate where the effect of roughness and vertical interfaces are seen. However, the correlation of these characterization techniques is not limited to these applications. Indeed, by using advanced operation modes of the AFM, maps of diverse physical properties of the sample can be obtained at the same time as the topography.

We have explored the combination of ToF-SIMS analysis with three AFM advanced modes: piezoresponse force microcopy (PFM), scanning capacitance microscopy (SCM) and scanning spreading resistance microscopy (SSRM). These operation modes respectively allow to map ferroelectric domains, to locally measure capacitance variations and to image the sample surface resistivity.

The combined ToF-SIMS / AFM methodology was applied ex-situ per individual AFM mode on diverse samples for applications focused on micro-electronics. We will present here some promising results highlighting the strength and the perspectives of the expansion of this combination to other applications.

[1] M.A. Moreno et al, J. Vac. Sci. Technol. B 36 (2018) 03F122.