Back to Projects

Team Name:

Porretas

Team Members:

Gildeci dos Santos Silva

Evidence of Work

sqly-me

Project Info

Team Name

Porretas

Team Members

2 members with unpublished profiles.

Project Description

Read the README description at the repository to better understand how it works.

However, a simple description is: When there is a need of using more than one dataset, there is a need to cross them at some point. This project is a Python Module capable of doing such merge automatically. It is an enabling tool to allow data scientists (for instance) to avoid the effort of doing it. When it merges all the structured data the user wants, it returns a report and a sqlite database file for the user to then proceed on their analysis.

This project proposes a solution for multiple challenges by creating a Python module capable of taking multiple .csv or .xslx files and combining them into one single database, on a sqlite file. In addition, it provides a report with all files that were able to merge into such database.
It is an enabling tool, not a data analysis and information giving software. However, it is capable of checking all possible combinations inside the total range of files given. For instance, if 10 files are given and two different clusters of data exists on them, it will generate 2 different databases alongside with their respective reports.

Data Story

All outcomes from the data used on this project is present at the Google Drive folder:
https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws?usp=sharing

Team DataSets

Digital Culture

Description of Use For this challenge, multiple datasets from this URL were used. Which ones were used and which ones successfully merged are presented at the report on the Google Drive Folder: https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws

Data Set

Physical Culture

Data Set

Households in Housing Stress

Description of Use Investing in Regions datasets used. Showcasing our Regions datasets. In addition to this one, these following datasets were used: https://data.sa.gov.au/data/dataset/households-in-25-housing-stress https://data.sa.gov.au/data/dataset/housing-stress-50-of-income The outcome is displayed at the team google drive folder: https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws

Data Set

TM-Link Data Discovery

Description of Use We crossed both TM-Link's application and applicants dataset to get an overview of the given data. The outcome is a report giving an analysis of that merge. However, the real outcome on the project is the capability of crossing any other dataset, as the user prefers. The report is found on the following link: https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws?usp=sharing

Data Set

Small Area Labour Markets (SALM) Data

Description of Use Small Area Labour Markets (SALM) Data redirects to the following page: https://employment.gov.au/small-area-labour-markets-publication From that page, two different .csv files (SA2 and LGA data tables) were downloaded and used on the project. The outcome is a report and a sqlite database file containing the united data of those and provided on the Google Drive folder, on the team details

Data Set

Challenge Entries

Investing in Regions

How might the Government prioritise its investment in regional South Australia for greatest regional development benefit?

Eligibility: Use any open dataset to support your entry.

Go to Challenge | 10 teams have entered this challenge.

Showcasing our regions

How might we promote South Australian regions to boost regional development?

Eligibility: Use any open dataset to support your entry.

Go to Challenge | 16 teams have entered this challenge.

Thrive or survive: how can we adapt for the future?

What will Australia in 2050 look like?

Eligibility: Must use one or more CSIRO datasets

Go to Challenge | 38 teams have entered this challenge.

Digital Culture

How do we make our digital cultural heritage collections engaging for online audiences? What experiences should we be developing beyond the search and retrieve box to visualise gallery, library and museum collections online and encourage their reuse and good storytelling?

Eligibility: Must use at least 2 datasets from 2 separate institutions that make up the North Terrace Cultural Precinct Innovation Lab

Go to Challenge | 14 teams have entered this challenge.

Physical Culture

How might we better integrate our digital collections and datasets into our physical gallery, library, or museum spaces?

Eligibility: Must use at least 2 datasets from 2 separate institutions that make up the North Terrace Cultural Precinct Innovation Lab

Go to Challenge | 9 teams have entered this challenge.

Australia@Sea: what is our future relationship with the ocean environment?

Our oceans are vital to the world’s economy and provide services for all Australians including food security, industries, tourism, and well-being.

Eligibility: Must use one or more CSIRO dataset

Go to Challenge | 17 teams have entered this challenge.

Australia’s Future Employment

Choose one of the following questions to address: 1. How can recent and future changes in the labour market help prepare young people for job opportunities? 2. What can we learn from case studies of regional labour markets? For example, what does rapid change in the industries or occupations within a region tell us about the needs of employers/workers in other regional labour markets

Eligibility: Use one or more datasets from the Labour Market Information Portal

Go to Challenge | 38 teams have entered this challenge.

TM-Link Data Discovery

TM-Link is a newly available trade mark database developed in collaboration between IP Australia, Swinburne University and Melbourne University. TM-Link includes administrative data from jurisdictions across the world, linked at the application level by advanced neural network algorithms. We are encouraging hackers to explore this new data set and consider what creative visualisations, innovative insights and/or opportunities to further enrich the data they might imagine.

Eligibility: Must use TM-Link

Go to Challenge | 8 teams have entered this challenge.

Local Government Information Technology Association of South Australia

How might we identify opportunities for improvement or new Council services, infrastructure and facilities to benefit community outcomes in South Australia?

Eligibility: Use of at least one Council dataset

Go to Challenge | 15 teams have entered this challenge.

Helping a social impact ‘start up’ (small organisation) to tell their story

Small and informal community/interest groups who have formed to solve local problems need data to know if their activities are making a difference and to re-design programs. How can we help these groups tell their story through data so they can seek support (political, financial, and on the ground) by showing how their programs are working, and decide where to focus next?

Eligibility: Use any open dataset to support your entry.

Go to Challenge | 23 teams have entered this challenge.

Back to Projects