Counting instances of genotype strings where order is irrelevant within locus

For each string, sort its characters, put them back together and count the frequencies with table.

fun  
    unlist(recursive = FALSE) |>
    lapply(sort) |> 
    sapply(paste, collapse = "") |>
    table(useNA = "ifany")
}
fun(offspring)
#> 
#> aabb aabB aaBB aAbb aAbB aABB AAbb AAbB AABB 
#>    1    2    1    2    4    2    1    2    1

^{Created on 2026-03-03 with reprex v2.1.1}

You can output as a data.frame by piping the above to as.data.frame.

fun(offspring) |> as.data.frame()
#>   Var1 Freq
#> 1 aabb    1
#> 2 aabB    2
#> 3 aaBB    1
#> 4 aAbb    2
#> 5 aAbB    4
#> 6 aABB    2
#> 7 AAbb    1
#> 8 AAbB    2
#> 9 AABB    1

^{Created on 2026-03-03 with reprex v2.1.1}

As @Rui Barradas mentioned, you could first make the sequence sorted and then apply table.

My solution follows the same idea but use order + intToUtf8 + utf8ToInt to finish the sorting processes

x 

canonicalizegenotype  
      sapply(paste, collapse='')
  } 
  x  
+   table()
aabbCC aaBbCc Aabbcc AabbCC AaBbcc AaBbCc AaBbCC AaBBcc AaBBCC AAbbcc AABbcc AABbCc AABBcc AABBCc AABBCC 
     1      1      1      1      1      1      2      1      1      1      1      1      1      1      1 

mkmat  
      paste(collapse='')
  }), simplify=FALSE) |> 
    sapply(paste, collapse='') |> 
    matrix(m, m)
}
set.seed(42)
df  
  as.data.frame()

A quick one-liner using Tmisc::strSort (same basic approach of sorting the strings before using table):

library(Tmisc)
table(strSort(unlist(offspring, 0, 0)))
#> 
#> aabb aabB aaBB aAbb aAbB aABB AAbb AAbB AABB 
#>    1    2    1    2    4    2    1    2    1

A faster option if performance is a concern. This uses the non-sorting unordered match approach found in this answer.

strtable