Posted in
134
6:55 am, August 15, 2022
Cars Data from Wiki and Autotrader
Created a manual sheet of car brands and models.
1300+ rows of brands and car names, current from autotrader.
Table / class created
cars.extend
PHP
public $load_array = [
"id",
"uid",
"insdate",
"title",
"additional",
"category",
"md5",
"processed",
"make",
"model",
"year_range",
"year_start",
"year_end",
"wiki_url",
"raw_table",
"assembly",
"body_style",
"image",
"images",
"description",
"manufacturer",
"also_called",
"production_years",
"model_years",
"designer",
"car_class",
"layout",
"platform",
"related",
"engines", /* need its own table? and link to cars.. */
"transmissions", /* need its own table? */
"kerb_weight",
"doors_number",
"power_output",
"wheelbase",
"car_length",
"car_width",
"car_height",
"predecessor",
"successor",
];Then it will need a wiki table, inserted in raw_table and the wiki link.
Create either an importer directly from the wiki, but i assume this will be bot blocked, so then manually paste in the table code, and an importer will need to load this table data to process the content.
Table Example.
HTML
<table class="infobox hproduct">
<tbody>
<tr>
<th colspan="2" class="infobox-above fn" style="font-size:125%; background-color:#C0C0C0; padding:0.25em 1em; text-align:center;">Abarth 124 Spider</th>
</tr>
<tr>
<td colspan="2" class="infobox-image"><a href="/wiki/File:2018_Abarth_124_Spider_Multiair_1.4.jpg" class="image"><img alt="2018 Abarth 124 Spider Multiair 1.4.jpg"
src="//upload.wikimedia.org/wikipedia/commons/thumb/5/5c/2018_Abarth_124_Spider_Multiair_1.4.jpg/280px-2018_Abarth_124_Spider_Multiair_1.4.jpg" decoding="async" width="280" height="158"
srcset="//upload.wikimedia.org/wikipedia/commons/thumb/5/5c/2018_Abarth_124_Spider_Multiair_1.4.jpg/420px-2018_Abarth_124_Spider_Multiair_1.4.jpg 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/5/5c/2018_Abarth_124_Spider_Multiair_1.4.jpg/560px-2018_Abarth_124_Spider_Multiair_1.4.jpg 2x"
data-file-width="4834" data-file-height="2726"></a></td>
</tr>
<tr>
<th colspan="2" class="infobox-header" style="background-color: #C0C0C0">Overview</th>
</tr>
<tr>
<th scope="row" class="infobox-label">Manufacturer</th>
<td class="infobox-data"><a href="/wiki/Mazda" title="Mazda">Mazda</a> <br> <a href="/wiki/FCA_Italy" title="FCA Italy">FCA Italy</a></td>
</tr>
<tr>
<th scope="row" class="infobox-label">Also called</th>
<td class="infobox-data">Fiat 124 Spider Abarth (North America)</td>
</tr>
<tr>
<th scope="row" class="infobox-label">Production</th>
<td class="infobox-data">2015–2019</td>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Model_year" title="Model year">Model years</a></th>
<td class="infobox-data">2017–2020</td>
</tr>
<tr>
<th scope="row" class="infobox-label">Assembly</th>
<td class="infobox-data">
<div class="plainlist">
<ul>
<li>Japan: <a href="/wiki/Hiroshima" title="Hiroshima">Hiroshima</a> (<a href="/wiki/Hiroshima_(Mazda_factory)" title="Hiroshima (Mazda factory)">Mazda Hiroshima plant</a>)</li>
<li>Italy: <a href="/wiki/Mirafiori" class="mw-redirect" title="Mirafiori">Mirafiori</a>, <a href="/wiki/Turin" title="Turin">Turin</a> (Officine Abarth, final assembly)</li>
</ul>
</div>
</td>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Automotive_design" title="Automotive design">Designer</a></th>
<td class="infobox-data">Ruben Wainberg<sup id="cite_ref-1" class="reference"><a href="#cite_note-1">[1]</a></sup></td>
</tr>
<tr>
<th colspan="2" class="infobox-header" style="background-color: #C0C0C0">Body and chassis</th>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Car_classification" title="Car classification">Class</a></th>
<td class="infobox-data"><a href="/wiki/Sports_car" title="Sports car">Sports car</a>/<a href="/wiki/Roadster_(automobile)" title="Roadster (automobile)">roadster</a> (<a href="/wiki/S-segment" title="S-segment">S</a>)</td>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Car_body_style" title="Car body style">Body style</a></th>
<td class="infobox-data">2-door <a href="/wiki/Convertible" title="Convertible">Convertible / Detachable hardtop (124 Spider Rally)</a></td>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Automobile_layout" class="mw-redirect" title="Automobile layout">Layout</a></th>
<td class="infobox-data"><a href="/wiki/Front-engine,_rear-wheel-drive_layout#Front_mid-engine,_rear-wheel-drive_layout" title="Front-engine, rear-wheel-drive layout">FMR layout</a></td>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Automobile_platform" class="mw-redirect" title="Automobile platform">Platform</a></th>
<td class="infobox-data"><a href="/wiki/List_of_Mazda_model_codes#Model_codes" title="List of Mazda model codes">Mazda NE/NF</a></td>
</tr>
<tr>
<th scope="row" class="infobox-label">Related</th>
<td class="infobox-data"><a href="/wiki/Fiat_124_Spider_(2016)" title="Fiat 124 Spider (2016)">Fiat 124 Spider (2016)</a><br><a href="/wiki/Mazda_MX-5_(ND)" title="Mazda MX-5 (ND)">Mazda MX-5 (ND)</a></td>
</tr>
<tr>
<th colspan="2" class="infobox-header" style="background-color: #C0C0C0">Powertrain</th>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Engine" title="Engine">Engine</a></th>
<td class="infobox-data">1.4 L <i><a href="/wiki/MultiAir" title="MultiAir">MultiAir</a></i> <a href="/wiki/Straight-four_engine" title="Straight-four engine">I4</a> (<a href="/wiki/Turbocharger" title="Turbocharger">turbocharged</a> <a
href="/wiki/Petrol_engine" title="Petrol engine">gasoline</a>)</td>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Transmission_(mechanics)" class="mw-redirect" title="Transmission (mechanics)">Transmission</a></th>
<td class="infobox-data category">
<div class="plainlist">
<ul>
<li>6-speed <a href="/wiki/Manual_transmission" title="Manual transmission">manual</a></li>
<li>6-speed <a href="/wiki/Automatic_transmission" title="Automatic transmission">automatic</a><sup id="cite_ref-autoblog_LA_2-0" class="reference"><a href="#cite_note-autoblog_LA-2">[2]</a></sup><sup id="cite_ref-fiatpress.com_tech_3-0"
class="reference"><a href="#cite_note-fiatpress.com_tech-3">[3]</a></sup></li>
</ul>
</div>
</td>
</tr>
<tr>
<th colspan="2" class="infobox-header" style="background-color: #C0C0C0">Dimensions</th>
</tr>
<tr>
<th scope="row" class="infobox-label"><a href="/wiki/Curb_weight" class="mw-redirect" title="Curb weight">Kerb weight</a></th>
<td class="infobox-data">1,060–1,080 kg (2,337–2,381 lb)<sup id="cite_ref-abarthpress.com_tech_4-0" class="reference"><a href="#cite_note-abarthpress.com_tech-4">[4]</a></sup></td>
</tr>
</tbody>
</table>
16 August 2022
Updated csv import and mapped the 1st 20 items and created custom title with make and model of car.
Import run ok.
PHP
public function import_csv($dir_file) {
global $functions;
global $bugs;
$db_table_name = $this->db->escapeString($this->db_table_name);
// $dir_file format : dir/file.csv :: csv/myfile.csv
if(!isset($dir_file)) {
$this->status = "Missing File for CSV Import";
return false;
}
// read the csv
$row = 1;
$out = "";
if (($handle = fopen($dir_file, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
$num = count($data);
$out .= "<p> $num fields in line $row: <br /></p>";
$row++;
$class = new $db_table_name; // call this class or another to add the csv items
/*
0 "id",
1 "uid",
2 "insdate",
3 "title",
4 "additional",
5 "category",
6"md5",
7"processed",
8"make",
9"model",
10"year_range",
11"year_start",
12"year_end",
13"wiki_url",
14"raw_table",
15"assembly",
16"body_style",
17"image",
18"images",
19"description",
20"manufacturer",
21"also_called",
22"production_years",
23"model_years",
24"designer",
25"car_class",
26"layout",
27"platform",
28"related",
29"engines",
30"transmissions",
31"kerb_weight",
32"doors_number",
33"power_output",
34"wheelbase",
35"car_length",
36"car_width",
37"car_height",
38"predecessor",
39"successor",
*/
// old manual method of assigning csv values to data
for ($c=0; $c < $num; $c++) {
$out .= $data[$c] . "<br />\n";
$var_name = $this->load_array[$c];
$this->$var_name = trim($data[$c]);
/*
Build the title... from make and model, 8 and 9.
Title
as this should run from 0-39 so need to do it in order.
*/
/* processed */
if($c == 7) {
$this->processed = trim($data[$c]);
$out .= "processed: " . $this->processed . "<br>";
}
/* processed */
/* Title in Parts */
/* make */
if($c == 8) {
$this->make = trim($data[$c]);
$out .= "make: " . $this->make . "<br>";
$title_parts = trim($data[$c]) . " ";
}
/* model */
if($c == 9) {
$this->model = trim($data[$c]);
$out .= "model: " . $this->model . "<br>";
$title_parts .= trim($data[$c]);
$this->title = $title_parts;
$this->md5 = md5($title_parts);
$out .= "title: " . $this->title . "<br>";
$out .= "md5: " . $this->md5 . "<br>";
}
/* Title in Parts */
/* year_range */
if($c == 10) {
$this->year_range = trim($data[$c]);
$out .= "year_range: " . $this->year_range . "<br>";
}
/* year_range */
/* year_start */
if($c == 11) {
$this->year_start = trim($data[$c]);
$out .= "year_start: " . $this->year_start . "<br>";
}
/* year_start */
/* year_end */
if($c == 12) {
$this->year_end = trim($data[$c]);
$out .= "year_end: " . $this->year_end . "<br>";
}
/* year_end */
/* wiki_url */
if($c == 13) {
$this->wiki_url = trim($data[$c]);
$out .= "wiki_url: " . $this->wiki_url . "<br>";
}
/* wiki_url */
/* raw_table */
if($c == 14) {
$this->raw_table = trim($data[$c]);
$out .= "raw_table: " . $this->raw_table . "<br>";
}
/* raw_table */
/* assembly */
if($c == 15) {
$this->assembly = trim($data[$c]);
$out .= "assembly: " . $this->assembly . "<br>";
}
/* assembly */
/* body_style */
if($c == 16) {
$this->body_style = trim($data[$c]);
$out .= "body_style: " . $this->body_style . "<br>";
}
/* body_style */
/* image */
if($c == 17) {
$this->image = trim($data[$c]);
$out .= "image: " . $this->image . "<br>";
}
/* image */
/* images */
if($c == 18) {
$this->images = trim($data[$c]);
$out .= "images: " . $this->images . "<br>";
}
/* images */
/* description */
if($c == 19) {
$this->description = trim($data[$c]);
$out .= "description: " . $this->description . "<br>";
}
/* description */
/*
if($c == 3) {
$this->title = $data[$c];
$out .= "title: " . $this->title . "<br>";
}
if($c == 0) {
$this->title = $data[$c]; // i think this should be this not class as in the examples below...
$out .= "title: " . $this->title . "<br>";
}
if($c == 1) { $class->phone_brand = trim($data[$c]); } // one line example with trim
if($c == 1) {
$class->data2 = trim($data[$c]);
$out .= "data2: " . $class->data2 . "<br>";
}
if($c == 2) {
$class->data3 = trim($data[$c]);
$out .= "gics_code: " . $class->data3 . "<br>";
}
*/
}
/* optional check for data title before adding */
if($this->title > " ") {
if($this->add()) {
$out .= "Added<br>";
}
}
}
fclose($handle);
}
return $out;
}
Now to test the table data import, and see how that may work per item.
..