How can Mercurial (or any other DVCS) recognize partially overlapped histories? E.g. fine grain / coarse grain.
Is there any way in Mercurial to usefully merge two repositories
where the lines of history are similar, but not identical?
E.g. where one rep has coarse grain revisions 0,1,2
and the other has fine grain revisions 0, 0.1, 0.2, 1, 1.1, 1.2, 2,
and come up with a single history?
Rather than a mess of branches and heads, which is what I get when I try using what I know of Mercurial?
Or the even fancier
Repo 1: 0, 1, 1.1, 1.2, 2
Repo 2: 0, 0.1, 0.2, 1, 2, 3
Merge: 0, 0.1, 0.2, 1, 1.1, 1.2 2, 3
In more detail
What I want is a merge that can recognize when file contents are the same,
or which can recognize that two lines of history are similar, although not all versions in one line
are in the other,
and give something like:
o=o changesets with same file contents on different historical lines
o | (line1)
| | changeset: 2:2a02e67e7b5d
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| o (line2)
| | changeset: 8:089179dde80a
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| |
| o changeset: 7:615416921e33
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.2
| |
| o changeset: 6:a43a88065141
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.1
| |
| |
o=o changesets with same file contents on different historical lines
o | (line1)
| | changeset: 1:93cbae111269
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:13 2012 -0700
| | summary: 1
| o (line2)
| | changeset: 5:fef4050e0162
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 1
| |
| o changeset: 4:b51fbedc72e5
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 0.2
| |
| o changeset: 3:45b7f64b2a23
| | parent: 0:c80bc10826be
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 0.1
| |
| |
|/
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
I can imagine that possibly a merge changeset would be necessary at the o=o points.
But I would like to have it recognized automatically.
Here's an example of how such a history would be created.
Contrived in this example, but something siomilar is happening to me in real-life,
when a project wants coarse grain commits, but where I want to preserve the fine grain ciommits
(as well as the coarse grain stuff released to the project).
[glew#mipscs587 ~/hack/hg-granularity] 900$ bash 12:39:54>. ./eg
% set verbose
% mkdir hg-repo
% cd hg-repo
% ./hg-repo
% hg init
% echo 0 > a
% hg add a
% hg ci -m0 a
% cd ..
% hg clone hg-repo fine
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
% hg clone hg-repo coarse
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
% cd fine
./fine
% echo 0.1 > a; hg ci -m0.1
% echo 0.2 > a; hg ci -m0.2
% echo 1 > a; hg ci -m1
% cat a
1
% hg push default
pushing to /home/glew/hack/hg-granularity/hg-repo
searching for changes
adding changesets
adding manifests
adding file changes
added 3 changesets with 3 changes to 1 files
% hg glog
# changeset: 3:fef4050e0162
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 1
|
o changeset: 2:b51fbedc72e5
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 0.2
|
o changeset: 1:45b7f64b2a23
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 0.1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
% cd ../coarse
% cp ../fine/a .
% cat a
1
% hg ci -m1
% hg glog
# changeset: 1:93cbae111269
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
% cd ../fine
% echo 1.1 > a; hg ci -m1.1
% echo 1.2 > a; hg ci -m1.2
% echo 2 > a; hg ci -m2
% cat a
2
% hg push default
pushing to /home/glew/hack/hg-granularity/hg-repo
searching for changes
adding changesets
adding manifests
adding file changes
added 3 changesets with 3 changes to 1 files
% hg glog
# changeset: 6:089179dde80a
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:15 2012 -0700
| summary: 2
|
o changeset: 5:615416921e33
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:14 2012 -0700
| summary: 1.2
|
o changeset: 4:a43a88065141
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:14 2012 -0700
| summary: 1.1
|
o changeset: 3:fef4050e0162
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 1
|
o changeset: 2:b51fbedc72e5
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 0.2
|
o changeset: 1:45b7f64b2a23
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 0.1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
% cd ../coarse
% cp ../fine/a .
% cat a
2
% hg ci -m2
% hg glog
# changeset: 2:2a02e67e7b5d
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:15 2012 -0700
| summary: 2
|
o changeset: 1:93cbae111269
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
OK, so now I have a fine grain history in the fine repo,
and a coarse grain history in the coarse repo. I wouldf like to merge them.
(Firget that the coarse is a subset of the fine: I can easily contrive exanples where they are not).
Simply pushing the coarse grain history gives a warning.
I will opush it later,
but first I will try merging in a separate clone.
% hg push default
pushing to /home/glew/hack/hg-granularity/hg-repo
searching for changes
abort: push creates new remote head 2a02e67e7b5d!
(you should pull and merge or use push -f to force)
% cd ..
% hg clone coarse merge-fine-and-coarse
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
% cd merge-fine-and-coarse/
./merge-fine-and-coarse/
% hg glog
# changeset: 2:2a02e67e7b5d
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:15 2012 -0700
| summary: 2
|
o changeset: 1:93cbae111269
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
% hg pull ../hg-repo
pulling from ../hg-repo
searching for changes
adding changesets
adding manifests
adding file changes
added 6 changesets with 6 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
% hg heads
changeset: 8:089179dde80a
tag: tip
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:15 2012 -0700
summary: 2
changeset: 2:2a02e67e7b5d
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:15 2012 -0700
summary: 2
Here is the merge.
Notice that the pairs
o changeset: 8:089179dde80a
| # changeset: 2:2a02e67e7b5d
and
o changeset: 5:fef4050e0162
| o changeset: 1:93cbae111269
have the same file contents,
one from the coartse and the other from the fine repo.
But the Mercurial history graph does not reflect this.
% hg glog
o changeset: 8:089179dde80a
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:15 2012 -0700
| summary: 2
|
o changeset: 7:615416921e33
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:14 2012 -0700
| summary: 1.2
|
o changeset: 6:a43a88065141
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:14 2012 -0700
| summary: 1.1
|
o changeset: 5:fef4050e0162
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 1
|
o changeset: 4:b51fbedc72e5
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 0.2
|
o changeset: 3:45b7f64b2a23
| parent: 0:c80bc10826be
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 0.1
|
| # changeset: 2:2a02e67e7b5d
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| |
| o changeset: 1:93cbae111269
|/ user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
% hg diff -r 2a02e67e7b5d -r 089179dde80a
So I'll try a merge
% hg merge -r 8
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
% hg ci -m'merge of fine and coarse at 2'
Better -
this shows that
o changeset: 8:089179dde80a
| # changeset: 2:2a02e67e7b5d
are a convergence point,
although an extra dummy changesrt was necessary.
But it does not show the commonality between
o changeset: 5:fef4050e0162
| o changeset: 1:93cbae111269
Here's the merged graph
% hg glog
# changeset: 9:328db8187d31
|\ tag: tip
| | parent: 2:2a02e67e7b5d
| | parent: 8:089179dde80a
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:43:51 2012 -0700
| | summary: merge of fine and coarse at 2
| |
| o changeset: 8:089179dde80a
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| |
| o changeset: 7:615416921e33
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.2
| |
| o changeset: 6:a43a88065141
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.1
| |
| o changeset: 5:fef4050e0162
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 1
| |
| o changeset: 4:b51fbedc72e5
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 0.2
| |
| o changeset: 3:45b7f64b2a23
| | parent: 0:c80bc10826be
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 0.1
| |
o | changeset: 2:2a02e67e7b5d
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| |
o | changeset: 1:93cbae111269
|/ user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
How about another merge?
% hg update -r 1
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
% hg merge -r 5
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
% hg ci -m'merge of fine and coarse at 1'
created new head
% hg glog
# changeset: 10:cca7fec90d3f
|\ tag: tip
| | parent: 1:93cbae111269
| | parent: 5:fef4050e0162
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:45:03 2012 -0700
| | summary: merge of fine and coarse at 1
| |
| | o changeset: 9:328db8187d31
| | |\ parent: 2:2a02e67e7b5d
| | | | parent: 8:089179dde80a
| | | | user: Andy Glew <glew#mips.com>
| | | | date: Thu Jun 21 12:43:51 2012 -0700
| | | | summary: merge of fine and coarse at 2
| | | |
| | | o changeset: 8:089179dde80a
| | | | user: Andy Glew <glew#mips.com>
| | | | date: Thu Jun 21 12:40:15 2012 -0700
| | | | summary: 2
| | | |
| | | o changeset: 7:615416921e33
| | | | user: Andy Glew <glew#mips.com>
| | | | date: Thu Jun 21 12:40:14 2012 -0700
| | | | summary: 1.2
| | | |
| +---o changeset: 6:a43a88065141
| | | user: Andy Glew <glew#mips.com>
| | | date: Thu Jun 21 12:40:14 2012 -0700
| | | summary: 1.1
| | |
| o | changeset: 5:fef4050e0162
| | | user: Andy Glew <glew#mips.com>
| | | date: Thu Jun 21 12:40:12 2012 -0700
| | | summary: 1
| | |
| o | changeset: 4:b51fbedc72e5
| | | user: Andy Glew <glew#mips.com>
| | | date: Thu Jun 21 12:40:12 2012 -0700
| | | summary: 0.2
| | |
| o | changeset: 3:45b7f64b2a23
| | | parent: 0:c80bc10826be
| | | user: Andy Glew <glew#mips.com>
| | | date: Thu Jun 21 12:40:12 2012 -0700
| | | summary: 0.1
| | |
+---o changeset: 2:2a02e67e7b5d
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| |
o | changeset: 1:93cbae111269
|/ user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
This is not right. It has established a new head, whereas
what we wanted was some way of indicating that
o changeset: 5:fef4050e0162
| o changeset: 1:93cbae111269
are the same.
OK, switch back to the original coarse
% cd ../coarse
% hg push default
pushing to /home/glew/hack/hg-granularity/hg-repo
searching for changes
abort: push creates new remote head 2a02e67e7b5d!
(you should pull and merge or use push -f to force)
% hg push -f default
pushing to /home/glew/hack/hg-granularity/hg-repo
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 1 files (+1 heads)
% hg glog
# changeset: 2:2a02e67e7b5d
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:15 2012 -0700
| summary: 2
|
o changeset: 1:93cbae111269
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
% cd ../hg-repo/
% hg glog
o changeset: 8:2a02e67e7b5d
| tag: tip
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:15 2012 -0700
| summary: 2
|
o changeset: 7:93cbae111269
| parent: 0:c80bc10826be
| user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:13 2012 -0700
| summary: 1
|
| o changeset: 6:089179dde80a
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| |
| o changeset: 5:615416921e33
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.2
| |
| o changeset: 4:a43a88065141
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.1
| |
| o changeset: 3:fef4050e0162
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 1
| |
| o changeset: 2:b51fbedc72e5
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 0.2
| |
| o changeset: 1:45b7f64b2a23
|/ user: Andy Glew <glew#mips.com>
| date: Thu Jun 21 12:40:12 2012 -0700
| summary: 0.1
|
# changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
% echo This is not right
This is not right
What I want is a merge that can recognize when file contents are the same,
or which can recognize that two lines of history are similar, although not all versions in one line
are in the other,
and give something like:
o=o changesets with same file contents on different historical lines
o | (line1)
| | changeset: 2:2a02e67e7b5d
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| o (line2)
| | changeset: 8:089179dde80a
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:15 2012 -0700
| | summary: 2
| |
| o changeset: 7:615416921e33
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.2
| |
| o changeset: 6:a43a88065141
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:14 2012 -0700
| | summary: 1.1
| |
| |
o=o changesets with same file contents on different historical lines
o | (line1)
| | changeset: 1:93cbae111269
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:13 2012 -0700
| | summary: 1
| o (line2)
| | changeset: 5:fef4050e0162
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 1
| |
| o changeset: 4:b51fbedc72e5
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 0.2
| |
| o changeset: 3:45b7f64b2a23
| | parent: 0:c80bc10826be
| | user: Andy Glew <glew#mips.com>
| | date: Thu Jun 21 12:40:12 2012 -0700
| | summary: 0.1
| |
| |
|/
|
o changeset: 0:c80bc10826be
user: Andy Glew <glew#mips.com>
date: Thu Jun 21 12:40:11 2012 -0700
summary: 0
I can imagine that possibly a merge changeset would be necessary at the o=o points.
But I would like to have it recognized automatically.
Heck, forget "recognized". I would like to have a way that I can recognize it manually, but have it represented in Mercurial.
I don't think Mercurial can get the design pattern you are looking for. Mercurial tracks the difference from a parent changeset to a child changeset so it doesn't really care about the contents of a file as a whole. Even if your file at revision 2 on tree A and the file at revision 5 on tree B have the same contents they were descended from different parents and thus have a different way of storing how they were changed. I haven't looked too much into the mq extension but that is the only thing I can think of that can edit a tree's history enough to do what you want. Editing a tree's history is always a dangerous thing to do and it is rarely recommended.
Edit:
I'm not too sure on this one either but the rebase extension might work for you as well.
Related
In Mercurial, is it possible to create a repository log which only only shows branch and merge history? i.e., any other commits would be excluded?
If possible, this would present a higher-level view of project history which may be useful in some circumstances (such as training others).
The ideal would be to have a graphical log output of this (such as in TortoiseHg).
Revsets are so powerful. May be showing the merge() changesets or its parents.
$ hg log -G -T "{rev} {branch}" -r "merge() or merge()^1 or merge()^2"
Gives something like
o 296 stable
|\
| o 295 test
| |\
| | # 294 default
| | |
o | | 292 stable
| | |
o | | 290 stable
|\| |
| o | 289 test
| |\|
| | o 288 default
| | |
| o | 287 test
| |\|
| | o 286 default
| | |
| o | 284 test
| |\|
| | o 283 default
...
o | 30 stable
|\|
o | 29 stable
| |
| o 28 default
| |
| o 27 default
|/|
o | 26 stable
|/
o 4 default
|
The only commit changesets are hidden.
repository log which only only shows branch and merge history?
Yes. hg help revsets
The ideal would be to have a graphical log output of this
hg log -G -r "REVSET"
I'm trying to use a public dataset in Amazon AWS, called Twilio/Wigle.net Street Vector Data Set. This dataset contains data of US street names and address ranges. Its size is about 10 GiB. When we take a look in linux, it looks like the following :
ubuntu#ip-172-31-xxx-xxx:/data-us-street$ pwd
/data-us-street
ubuntu#ip-172-31-xxx-xxx:/data-us-street$ ll
total 20576
drwxr-xr-x 6 27 sudo 4096 May 19 2009 ./
drwxr-xr-x 23 root root 4096 Apr 20 18:10 ../
-rw-r--r-- 1 root root 8339 May 19 2009 README
drwx------ 2 27 sudo 4096 Mar 18 2009 addresses/
-rw-rw---- 1 27 sudo 5242880 Mar 18 2009 ib_logfile0
-rw-rw---- 1 27 sudo 5242880 Mar 8 2009 ib_logfile1
-rw-rw---- 1 27 sudo 10485760 Mar 18 2009 ibdata1
drwx------ 2 27 sudo 16384 Mar 8 2009 lost+found/
drwx------ 2 27 sudo 4096 Mar 8 2009 mysql/
-rw-rw---- 1 27 sudo 117 Mar 18 2009 mysql-bin.000001
-rw-rw---- 1 27 sudo 19 Mar 18 2009 mysql-bin.index
drwx------ 2 27 sudo 4096 Mar 8 2009 test/
In order to use its data, I want to link it to a MySQL database in the same host. Can somebody tell me how to do it ?
What I've tried
I've tried to overwrite the mysql storage directory datadir located at /etc/mysql/my.cnf with the following change :
#datadir = /var/lib/mysql
datadir = /data-us-street
I stopped the server, changed to value then restarted the MySQL server. However, it doesn't work.
README
/*
Copyright (c) 2009 Twilio, Inc.
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
*/
1. The Twilio/Wigle.net public address dataset
This public dataset contains the street data for the U.S., based on
the geodata published by the U.S. Census Bureau's TIGER project.
We have reformated the data from GIS friendly shapefiles to a more
generally accessible MySQL database.
This dataset covers all the streets, roads, and highways in the U.S.
These streets are represented as "polylines" which are shapes made up
of individual line segments. Each polyline has its own unique ID,
and each segment that makes up the polyline has a sequence number.
The combination of this ID and sequence number is the primary key of
the address table. Each row in the database represents one of these
line segments.
2. Description of Columns
+-------------+------------+------+-----+------------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------+------+-----+------------+-------+
| id | char(10) | NO | PRI | | |
| seq | char(3) | NO | PRI | | |
| name | char(30) | YES | MUL | NULL | |
| prefix | char(2) | YES | MUL | NULL | |
| type | char(4) | YES | | NULL | |
| startlat | float(12,8)| NO | | 0.00000000 | |
| startlong | float(12,8)| NO | | 0.00000000 | |
| endlat | float(12,8)| NO | | 0.00000000 | |
| endlong | float(12,8)| NO | | 0.00000000 | |
| leftzip | int(5) | YES | MUL | NULL | |
| rightzip | int(5) | YES | MUL | NULL | |
| leftaddr1 | char(11) | YES | | NULL | |
| leftaddr2 | char(11) | YES | | NULL | |
| rightaddr1 | char(11) | YES | | NULL | |
| rightaddr2 | char(11) | YES | | NULL | |
| name_dtmf | char(30) | YES | | NULL | |
| prefix_dtmf | char(2) | YES | | NULL | |
+-------------+------------+------+-----+------------+-------+
id - this is the unique ID of a street, road, or highway polyline,
according to US census blocks
seq - this is the sequence number of this segment in the polyline,
used to correlate 2 segments of the same street within a US
census block
name - the name of this street, road, or highway. example: Main
prefix - the prefix of the street name. examples: N,S,E,W
type - the suffix of the street name. examples: Blvd, St, Rd
startlat/startlong - the lat/long pair of this segments the starting
point
endlat/endlong - the lat/long pair of this segments the ending point
leftzip/rightzip - the zipcode of the addresses on the corresponding
side of the street
leftaddr1/leftaddr2/rightaddr1/rightaddr2 - the starting and ending
addresses
numbers for each side of the street
name_dtmf/prefix_dtmf/type_dtmf - the name, prefix and type columns,
represented
as DTMF encoded numbers. For example, (A,B,C) = 2, (D,E,F) = 3,
etc.
Useful for telephony applications.
3. Example Segment
--------------------------------------------------------
id: 111710515
seq: 0
name: Main
prefix:
type: St
startlat: 41.49493408
startlong: -87.70324707
endlat: 41.49483490
endlong: -87.70324707
leftzip: 60466
rightzip: 60443
leftaddr1: 21801
leftaddr2: 21805
rightaddr1: 21800
rightaddr2: 21804
name_dtmf: 6246
prefix_dtmf:
--------------------------------------------------------
Name: Main St.
Left Zip: 60466
21801 21805
X------------------------------------>
21800 21804
Right Zip: 60443
4. Common Queries
a. Find the street segment record for 30 Rockefeller Plaza, NY, NY,
10020
SELECT * FROM address WHERE leftzip=10020 AND name LIKE 'Rockefeller'
AND 30 BETWEEN leftaddr1 AND leftaddr2
UNION
SELECT * FROM address WHERE rightzip=10020 AND name LIKE
'Rockefeller' AND 30 BETWEEN rightaddr1 AND rightaddr2
--------------------------------------------------------
id: 59657155
seq: 0
name: Rockefeller
prefix:
type: Plz
startlat: 40.7585
startlong: -73.979
endlat: 40.7592
endlong: -73.9786
leftzip: 10020
rightzip: 10020
leftaddr1: 22
leftaddr2: 38
rightaddr1: 21
rightaddr2: 39
name_dtmf: 7625335537
prefix_dtmf:
--------------------------------------------------------
b. Find all the street names within a zipcode.
SELECT DISTINCT prefix, name, type FROM address WHERE leftzip=10009
OR rightzip = 10009
+--------+----------+------+
| prefix | name | type |
+--------+----------+------+
| | 1st | Ave |
| | Avenue A | |
| E | 13th | St |
| E | 14th | St |
| E | 12th | St |
| E | 11th | St |
| E | 10th | St |
| E | 2nd | St |
| E | 3rd | St |
| E | 4th | St |
| E | 6th | St |
| E | 7th | St |
| | St Marks | Pl |
| E | 9th | St |
| | Avenue B | |
| E | 5th | St |
| | Avenue C | |
| E | 20th | St |
| E | 15th | St |
| E | 16th | St |
| E | 8th | St |
| | Avenue D | |
| | Szold | Pl |
+--------+----------+------+
c. Streets starting with F in a zipcode
SELECT DISTINCT prefix, name, type from address WHERE leftzip = 94117
AND name LIKE 'F%'
UNION
SELECT DISTINCT prefix, name, type from address WHERE rightzip =
94117 AND name LIKE 'F%'
+--------+------------+------+
| prefix | name | type |
+--------+------------+------+
| | Fulton | St |
| | Fell | St |
| | Frederick | St |
| | Farnsworth | Ln |
| | Fillmore | St |
| | Friendship | Ct |
+--------+------------+------+
d. Streets starting with "FRE" as DTMF digits in a zipcode:
SELECT DISTINCT prefix, name, type from address WHERE leftzip = 94117
AND name_dtmf LIKE '373%'
UNION
SELECT DISTINCT prefix, name, type from address WHERE rightzip =
94117 AND name_dtmf LIKE '373%'
+--------+------------+------+
| prefix | name | type |
+--------+------------+------+
| | Fulton | St |
| | Fell | St |
| | Frederick | St |
| | Farnsworth | Ln |
| | Fillmore | St |
| | Friendship | Ct |
+--------+------------+------+
Every sub-folder under the root folder /data-us-street represents a database. So if I want to use database addresses, I need to use folder ./addresses:
Copy the target database folder into the data directory of the current MySQL Database Server instance, e.g. /var/lib/mysql
$ cp -r /data-us-street/addresses /var/lib/mysql/addresses
Change the folder owner recursively
$ chown mysql:mysql /var/lib/mysql/addresses/*
Use database addresses and enjoy !
is there a mysql date function that provides yyyy-mm-ww dates based on 53 weeks of the year as per the table below? For example, 2015-01-01 is (2015-jan-week01). Or, is there a method you'd recommend that would allow me to achieve yyyy-mm-ww?
btw, is there a term for this type of month grouping by week? i didn't know what to search for
jan feb mar
--------------- -------------- ----------------
| 01 02 03 04 | | 05 06 07 08| |09 10 11 12 13 |
apr may jun
--------------- -------------- -----------------
| 14 15 16 17 | | 18 19 20 21| |22 23 24 25 26 |
jul aug sep
--------------- -------------- -----------------
| 27 28 29 30 | | 31 32 33 34| | 35 36 37 38 39 |
oct nov dec
--------------- -------------- --------------------
| 40 41 42 43 | | 44 45 46 47| | 48 49 50 51 52 53 |
Several modes of MySQL's WEEK() function use 53 weeks in a year, see http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_week
mysql> SELECT WEEK('2008-12-31',1);
+----------------------+
| WEEK('2008-12-31',1) |
+----------------------+
| 53 |
+----------------------+
1 row in set (0.00 sec)
mysql> SELECT CONCAT_WS('-', YEAR('2008-12-31'), SUBSTRING(MONTHNAME('2008-12-31'), 1, 3), WEEK('2008-12-31',1));
+----------------------------------------------------------------------------------------------------+
| CONCAT_WS('-', YEAR('2008-12-31'), SUBSTRING(MONTHNAME('2008-12-31'), 1, 3), WEEK('2008-12-31',1)) |
+----------------------------------------------------------------------------------------------------+
| 2008-Dec-53 |
+----------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> SELECT CONCAT(YEAR('2015-01-01'), '-', LOWER(SUBSTRING(MONTHNAME('2015-01-01'), 1, 3)), '-week', WEEK('2015-01-01',1));
+-----------------------------------------------------------------------------------------------------------------+
| CONCAT(YEAR('2015-01-01'), '-', LOWER(SUBSTRING(MONTHNAME('2015-01-01'), 1, 3)), '-week', WEEK('2015-01-01',1)) |
+-----------------------------------------------------------------------------------------------------------------+
| 2015-jan-week1 |
+-----------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>
I have the following data in my webinar_timing table in mysql database
start_time and end_time are of type datetime
id | webinar_id | start_time | end_time
-------------------------------------------------------------------
1 | 5 | 3/18/2015 6:00:00 PM | 3/18/2015 7:00:00 PM
2 | 5 | 3/19/2015 6:00:00 PM | 3/19/2015 7:00:00 PM
3 | 5 | 3/20/2015 6:00:00 PM | 3/20/2015 7:00:00 PM
4 | 5 | 3/21/2015 6:00:00 PM | 3/21/2015 7:00:00 PM
5 | 5 | 3/22/2015 6:00:00 PM | 3/22/2015 7:00:00 PM
6 | 11 | 3/20/2015 8:00:00 PM | 3/20/2015 9:00:00 PM
7 | 11 | 3/21/2015 8:00:00 PM | 3/21/2015 9:00:00 PM
8 | 11 | 3/22/2015 8:00:00 PM | 3/22/2015 9:00:00 PM
9 | 22 | 3/25/2015 8:00:00 PM | 3/25/2015 9:00:00 PM
10 | 22 | 3/27/2015 8:00:00 PM | 3/27/2015 9:00:00 PM
11 | 22 | 3/29/2015 8:00:00 PM | 3/27/2015 9:00:00 PM
Basically, for each webinar, I want the total occurences and number of classes completed or remaining AND the NEXT upcoming class
Egs: When I run this query say at 3/21/2015 at 4:00 PM - this is the result I am expecting
webinar_id | total | Classes Completed | Next Class
----------------------------------------------------------
5 | 5 | 3 | 3/21/2015 6:00:00 PM
11 | 3 | 1 | 3/21/2015 8:00:00 PM
22 | 3 | 0 | 3/25/2015 8:00:00 PM
OR
webinar_id | total | Classes Remaining | Next Class
----------------------------------------------------------
5 | 5 | 2 | 3/21/2015 6:00:00 PM
11 | 3 | 2 | 3/21/2015 8:00:00 PM
22 | 3 | 3 | 3/25/2015 8:00:00 PM
Any help will be appreciated
Thanks in advance
Just use SUM(IF(...))
select webinar_id, count(*) AS total,
SUM(IF(end_time<NOW(), 1, 0)) AS completed,
SUM(IF(start_time>=NOW(), 1, 0)) AS remaining
from webinar_times
group by webinar_id;
You'll have to figure out whether you are looking at start_time or end_time and what to do with equals and little details like that...
SELECT COUNT(*), W.webinar_id, X.COMPLETED, Y.[NOT COMPLETE]
FROM #Webinars W
JOIN (SELECT COUNT(*) AS 'COMPLETED', webinar_id
FROM #Webinars
WHERE end_tim <= '3/18/2015 7:00:00'
GROUP BY webinar_id) X
ON X.webinar_id = W.webinar_id
JOIN (SELECT COUNT(*) AS 'NOT COMPLETE', webinar_id
FROM #Webinars
WHERE end_tim > '3/18/2015 7:00:00'
GROUP BY webinar_id) Y
ON Y.webinar_id = W.webinar_id
GROUP BY W.webinar_id, X.COMPLETED, Y.[NOT COMPLETE]
I am trying to get a list of guests who have not yet fully paid their total charges (those who still have a balance).
I want to display like so:
1231231(reservation id) - John Doe(guest name) 2013-11-17(date due) 153.14(balance due)
without the ( ) of course.
I have tinkered but still can't find a query that is satisfactory.
Here is a link to the SQLFiddle.
If I haven't provided enough info, kindly let me know.
Are you looking for something like this?
SELECT r.id,
CONCAT(g.fname, ' ', g.lname) guest_name,
r.arrival date_due,
r.total_price - COALESCE(p.payment_amt, 0) balance_due
FROM reservations r JOIN guests g
ON r.guest = g.id LEFT JOIN
(
SELECT reservation, SUM(payment_amt) payment_amt
FROM payments
GROUP BY reservation
) p
ON r.id = p.reservation
HAVING balance_due > 0
Sample output (based on provided sample data):
| ID | GUEST_NAME | DATE_DUE | BALANCE_DUE |
|---------|------------|---------------------------------|-------------|
| 1000023 | John Doe | November, 09 2013 00:00:00+0000 | 40.5 |
| 1000022 | John Doe | November, 09 2013 00:00:00+0000 | 40.5 |
| 1000018 | John Doe | November, 01 2013 00:00:00+0000 | 54 |
| 1000019 | John Doe | November, 08 2013 00:00:00+0000 | 54 |
| 1000020 | Mary Jane | November, 08 2013 00:00:00+0000 | 54 |
| 1000021 | Mary Jane | November, 08 2013 00:00:00+0000 | 54 |
Here is SQLFiddle demo