New data release v1.0.0, which includes the following news files or edits:
import_microdata22(), which brings the
controlled-access microdata of the 2022 Population
Census into censobr. See new features below.weight_household and
hh_income are stored as integers. Totals weighted by
weight_household therefore differ slightly from earlier
releases – sum(weight_household) goes from 17,682,112 to
17,643,387 (-0.22%).filter(V0601 == "1") becomes
filter(V0601 == 1). Zero-padded codes lose the padding as
well: V0402 == "01" becomes V0402 == 1. Values
that used to be recorded as "." are now
NA.New features
import_microdata22() to import to censobr
the controlled-access microdata of 2022. Once the .zip file
with the original data is imported read_(year = 2022)
functions always read the controlled-access microdata. If the
controlled-access microdata have not been imported yet, these functions
return an informative warning and download the public microdata set,
which has fewer variables. See the new vignette Working
with 2022 microdata. Closes #79.add_labels = "pt" now works for all years and tables
since 1960. Closes #25, #26, and #27.merge_households parameter now works for all census
years since 1960. Because merging all ~300 population + househols
columns can require more than 20GB of memory,
read_population(merge_households = TRUE) requires
columns to be set – naming the columns you need
keeps the operation to a few seconds and a few dozen MB. Closes #31.Major changes
data_dictionary() now takes only two values in
dataset: "microdata", which opens a single
Excel file covering every variable of the microdata and is now available
for all censuses since 1960, and "tracts",
available since 1970.Breaking changes
year now require the user to
declare it. Previously questionnaire() silently assumed
year = 2010.questionnaire(type),
read_tracts(dataset) and
data_dictionary(dataset) are now explicitly required, and
the error message lists the values accepted.data_dictionary() now takes only two values in
dataset: "microdata", which opens a single
Excel file covering every variable of the microdata. The per-data-set
dictionaries opened with dataset = "population" and
dataset = "households" were retired, since the microdata
dictionary now covers the pre-2000 censuses too.Minor changes
data_dictionary(), questionnaire() and
interview_manual() now return the path to the downloaded
file. The file is only opened when verbose = TRUE and the
session is interactive, so scripted runs no longer launch a viewer.columns now only accepts a character vector of column
names, in all five microdata readers (read_population(),
read_households(), read_families(),
read_mortality(), read_emigration()), matching
its documented type. It previously also silently accepted numeric column
indices.dataset in data_dictionary()
is also case insensitive now, as in read_tracts().bug fixes
add_labels = "pt" in multiple read_ functions.add_labels = "pt" now compares the codes of every
census year as numbers, matching how the microdata are stored since the
v0.7.0 data release.year now returns an informative
error. Previously a vector such as year = c(2000, 2010)
failed with a cryptic “the condition has length > 1” message from
base R.NULL, so that
running it again downloads a fresh copy instead of failing on every
call.cache = FALSE no longer fails when the cache
directory does not exist yet, for example on a fresh installation.merge_households = TRUE is now removed when the merge
finishes. Previously it was left behind in the session’s temp
directory.censobr_cache(delete_file = 'old'). Set
options(censobr.keep_old_cache = TRUE) to keep those files,
for example to go on working with an older data release. Microdata
imported with import_microdata22() are never deleted,
because censobr cannot download them again.data_dictionary() now does not open the
file when verbose = FALSE. Closes 72get_censobr_cache_dir()set_censobr_cache_dir() now sets cache
directories that persist across R sessions. Closes #55. The data is
saved in versioned directory inside the cache directory.year parameter no longer defaults to
2010.verbose (logical) indicating whether
functions should print messsagesmerge_households = TRUE to
avoid duplicated columns.parquet compressed using
compression='zstd' and compression_level = 22.
This has almost halved the size of data files, making downloads much
more efficient at minimal cost of reading time.merge_households = TRUE now returns
the expected result.read_mortality,
read_emigration) now include a new parameter
merge_households (logical) to indicate whether the function
should merge household variables to the output data. Partially closes #31curl::multi_download() to download files in
parallel. This brings the advantage that the package now automatically
detects whether the data/documentation file has been updated and should
be downloaded again.ImportsImports to
Suggests while the {arrow} team fixes their conflict with
CRAN policies related to downloading binary software. See here.questionnaire() function now accepts questionnaires
of type: "long" or "short"..onAttach by .onLoad so that the
package works with censobr::function()read_tracts() to read Census tract-level
aggregate data.data_dictionary() opens on a browser the
data dictionary of Brazil’s census data.questionnaire() opens on a browser the
questionnaire used in the data collection of Brazil’s censuses.interview_manual() opens on a browser the
interview manual of the data collection of Brazil’s censuses.
set_censobr_cache_dir() that allows users
to set custom directory for caching files from the censobr package.censobr_cache(delete_file = 'all') now removes
all data and directories related from censobr.