Sqoop incremental import and update does not work - mysql

Question

How can I update the data in HDFS file similar to data in MySQL table?
I checked the internet, but all the examples given are with --incremental lastmodified example.
Where in my case my MySQL table does not contain a date or timestamp column.
How can I update the data in HDFS file similar to data in MySQL table that does not contain date column?
I have MySQL table as below
mysql> select * from employee;
+----+--------+--------+------+-------+-----------+
| id | name | gender | age | state | language |
+----+--------+--------+------+-------+-----------+
| 1 | user1 | m | 25 | tn | tamil |
| 2 | user2 | m | 41 | ka | tamil |
| 3 | user3 | f | 47 | kl | tamil |
| 4 | user4 | f | 52 | ap | telugu |
| 5 | user5 | m | 55 | ap | telugu |
| 6 | user6 | f | 43 | tn | tamil |
| 7 | user7 | m | 34 | tn | malayalam |
| 8 | user8 | f | 33 | ap | telugu |
| 9 | user9 | m | 36 | ap | telugu |
I imported to HDFS using the below command.
[cloudera#localhost ~]$ sqoop import --connect jdbc:mysql://localhost:3306/mydatabase --username root --table employee --as-textfile --target-dir hdfs://localhost.localdomain:8020/user/cloudera/data/employee
The data is imported as expected.
[cloudera#localhost ~]$ hadoop fs -ls /user/cloudera/data/employee/
Found 6 items
-rw-r--r-- 3 cloudera cloudera 0 2017-08-16 23:57 /user/cloudera/data/employee/_SUCCESS
drwxr-xr-x - cloudera cloudera 0 2017-08-16 23:56 /user/cloudera/data/employee/_logs
-rw-r--r-- 3 cloudera cloudera 112 2017-08-16 23:56 /user/cloudera/data/employee/part-m-00000
-rw-r--r-- 3 cloudera cloudera 118 2017-08-16 23:56 /user/cloudera/data/employee/part-m-00001
-rw-r--r-- 3 cloudera cloudera 132 2017-08-16 23:56 /user/cloudera/data/employee/part-m-00002
-rw-r--r-- 3 cloudera cloudera 136 2017-08-16 23:56 /user/cloudera/data/employee/part-m-00003
Now I updated values and inserted values in mysql table. But this table doesnot contain date column.
mysql> update employee set language = 'marathi' where id >= 8;
mysql> insert into employee (name,gender,age,state,language from people) values('user11','f','25','kl','malayalam');
I know the newly inserted values can be inserted to hdfs using --check-column, incremental append and --last-value.
But how can I update the values in hdfs for the mysql table rows 8 and 9 that were updated to 'marathi'? Also, my employee table does not contain a date or timestamp column.

Answer 1

For newly inserted row, you can always use:
--incremental append --check-column id --last-value 9
But for getting updates from table not having updated_at column, I don't think thats possible. If your table is very small, then probably just do a full dump every time.
Or if you somehow can maintain track of what all ids got updated since last import, then let's say you know ids 7, 3, 4 and 8 got updated since last import, you can use the minimum of updated ids and use as --last-value. So your config will be:
--incremental append --check-column id --last-value 3 --merge-key id
where --merge-key id will tell sqoop to merge the new incremental data with old based on id column.