July 01, 2014

Grey Gandalf

DataBasin: more powerful CSV writing for Select and Select-Identify

Grey Gandalf

<a href="https://gap.nongnu.org/databasin/index.html">DataBasin</a>, the clean-room impelmentation of a data extraction tool for <a href="https://www.salesforce.com/">SalesForce.com</a> available for <a href="https://www.gnustep.org/">GNUstep </a>and Mac just made a big leap forward!<br /><br />If you have ever done a SELECT in salesforce.com, you might have noticed that the results are not ordered and that semi-joined fields ("." notation) are handled strange because a whole object is returned.<br /><br />DataBasin extracts the &nbsp;columns of the CSV file by checking the first row of the dataset and recursing on the field names of salesforce's response. The effects in DataBasin are the following:<br /><div><ul><li>The field order is not preserved</li><li>If a sub-object is queried, all those fields are grouped together in that object</li><li>If more than one field is queried on a sub-object and on certain record this object is missing, the number of columns between records is inconsistent: salesforce.com doesn't return the whole object and DataBasin iterates record after record in the sub-object, but if it it is totally missing, only one empty column will be written, not as many as needed</li><li>Capitalization of the column names doesn't get preserved (compared to the above issues, something really minor)</li></ul><div><br /></div></div>I added a new option available in write out fields in the order of the query.<br /><br />This feature implies a more complex under the hood. For each record gotten in the response, the field names are reconstructed as before (e.g. by recursing inside sub-objects) and put in a Dictionary.<br /><br />The original SOQL query is parsed and the field names extracted, then these names are used as keys for each record dictionary and written out.<br /><div><ul><li>Order preserved, also with sub-objects</li><li>Fields preserved and empty values are written even if whole sub-objects are missing</li><li>Case is preserved</li></ul><div>This approach looks fine and comes with a relatively small performance penalty, however it is delicate because during parsing of the query, many naming subtleties of Salesforce.com needs to be inferred, that is, DataBasin tries to infer how the name in the Response will be:</div></div><div><ul><li><div>field aliases</div></li><li><div>aggregate names with no alias, which are called progressively Expr0, Expr1</div></li><li><div>idiosyncrasies with aggregate functions. A field in a sub-objects is called Object__c.Field__c when in a regular query, but when used in an aggregate query, only Field__c is used, disregarding the join-depth</div></li><li><div>COUNT() vs.COUNT(Id)</div></li></ul>I hope I have not missed any! in any case, enjoy!</div><br /><div class="separator"><a href="https://4.bp.blogspot.com/-lcncspnOY44/U61qKQoVVFI/AAAAAAAAA6Q/0wTfVbxSd3A/s1600/select_query_order.jpg"><img alt="" border="0" src="https://4.bp.blogspot.com/-lcncspnOY44/U61qKQoVVFI/AAAAAAAAA6Q/0wTfVbxSd3A/s1600/select_query_order.jpg" height="220" title="Select and Select identify" width="400" /></a></div><br /><br />This feature is available now both in Select and Select-Identify, since it is a problem of the CSV Write backend and enhances DataBasin operation usability greatly, bringing it on par with DataLoader.<br />Since DataLoader performces similar transformation and sometimes produces wrong results, I left this feature as optional with the flag "Write fields in query order".

July 01, 2014 02:24 AM

June 06, 2014

Gregory Casamento

GNUstep's position on Swift

Gregory Casamento

<div class="separator"><a href="https://4.bp.blogspot.com/-C0LZT_nVEnE/U5I--COHV0I/AAAAAAAABrk/1lYoLVMsvn0/s1600/swittol.png"><img border="0" src="https://4.bp.blogspot.com/-C0LZT_nVEnE/U5I--COHV0I/AAAAAAAABrk/1lYoLVMsvn0/s1600/swittol.png" height="300" width="400" /></a></div><div class="separator"><br /></div>I wanted to make this post to make it clear to the community regarding GNUstep's position on the new Swift language. &nbsp; &nbsp;If the language is released as open source then GNUstep will fully support it. &nbsp;If it is, however, not released as open source then we will either take steps to create an implementation ourselves or provide any assistance needed to a group of people other than ourselves who are willing to take that on.<br /><br />I believe that the language itself is a good one and that it represents an opportunity for more people to take part in the OSX, iOS and GNUstep communities if it's properly done.<br /><br />My previous post was to illustrate the dangers of lock-in if that is, indeed, Apple's intention. &nbsp;Those warnings should not be construed as any condemnation of the language itself. &nbsp;This post is not backpedaling, only a clarification of my previous position (see the comments in the previous post).<br />

June 06, 2014 04:22 PM

June 05, 2014

Gregory Casamento

Swift

Gregory Casamento

<br /><div class="separator"><a href="https://2.bp.blogspot.com/-nLIcBt79DI8/U5AKte52lII/AAAAAAAABrQ/Ecy5l-dQ1mk/s1600/Apple-Swift-Programming-Language.jpg"><img border="0" src="https://2.bp.blogspot.com/-nLIcBt79DI8/U5AKte52lII/AAAAAAAABrQ/Ecy5l-dQ1mk/s1600/Apple-Swift-Programming-Language.jpg" height="223" width="320" /></a></div><div class="separator"><br /></div><div class="separator">Swift is a new programming language developed by Apple as a replacement for Objective-C. &nbsp; I've had some time to review the language by reading the iBook which Apple made available on it for free. &nbsp; It seems to me that the language is very javascript &nbsp;like in it's design and is a clear concession to those who don't like Objective-C's syntax. &nbsp; The real purpose behind this language, however, is a little darker. &nbsp;I believe that it's ultimate purpose is lock-in. &nbsp; The more developers start using Swift the less they are going to be able to move to other platforms (such as Android). &nbsp;&nbsp;</div><div class="separator"><br /></div><div class="separator">Please be cautious when using it and make sure that the only platform you wish to release on is iOS because that is very likely the decision you ARE making if you choose swift.</div><div class="separator"><br /></div><div class="separator">The language itself is a very nice language, the implications of it are what concern me. &nbsp;Anything which impacts user/developer freedoms makes me concerned.</div>

June 05, 2014 12:20 AM

May 08, 2014

Grey Gandalf

Tailoring OpenBSD for an old strange computer

Grey Gandalf

I have an ol' OmniBook 800CT. A small, interesting computer, for its time, extremely advanced!<br />Small form factor, but still a very nice keyboard, something unmatched on modern netbooks. The unique pop-out mouse. The series started out with 386 processor, b&amp;w display and ROM expansions.<br />The 800CT is one of the latest models: same form factor, SCSI connector, but color screen (800x600) and a hefty Pentium 133Mhz!<br />But only 32 MB of ram (the kernel report 31 of real mem, 24 avail mem)<br /><br />Original 5.4 kernel: 9.2M<br />Custom kernel: 5.0 M<br /><br />This shrinkage is quite hefty! almost 50%! More than raw disk usage, this new kernel boots faster and leaves &nbsp;more free memory. Enough more that X11 is now almost usable<br /><br />How can this be achieved? essentially by removing unused kernel options. If you remove drivers which you know you don't need because you don't have the hardware (and won't use it, e.g. you know you won't plug-in a certain card in the future) then you configure it out, it won't be built and it won't get in your kernel.<br />On an old laptop with no expansion except the ports and the PCMCIA port it has, this is relatively easy.<br /><br />To build your custom kernel, follow the <a href="https://openbsd.org/faq/faq5.html#Options">OpenBSD FAQ</a>. <br /><br />The main theory is to take the kernel configuration file, skim over it line by line it and see if you have the hardware, which you know by checking your dmesg. Dmesg shows which devices and drivers were loaded.Remember that you do not modify GENERIC, but a copy of it.<br /><br />You can automate this with a tool called <span>dmassage</span>: it will parse your GENERIC configuration and produce an optimal tuned version, however it will not work out of the box.<br />Why? there are drivers which do not compile if other drivers are not present.<br /><br />I'm unsure if this is really a bug, in my opinion it is at least "unclean" code, however since mostly this kind of extreme driver-picking is not done, it is not fatal and probably won't be fixed.<br /><br />&nbsp;If you remove all drivers at once, you won't easily find out one which one breaks, so my suggestion is to remove them in sets. One by-one is surely too tedious, since for each you need to make a build.<br /><ol><li>remove X drivers</li><li>build, if it works, copy the configuration file as a backup</li><li>test the kernel, optionally, by booting it</li><li>continue removal</li></ol><br />Thus, in case of breakage, you can narrow it down to a less options.<br /><br />If your mahcine doesn't have a certain bus, you may remove all drievrs attached to each. But proceed from the leaves, not the trunk: gradually remove the peripheral drivers before removing the bus support.<br /><br />In my case, I found that an unremovable driver is:<br /><span>et*&nbsp;&nbsp;&nbsp; at pci?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; # Agere/LSI ET1310 </span><br /><br /><br />Remember that you are running an <i>unsupported</i> kernel, if you want support for a problem, better try it with the original kernel, of which you should anyway for safety retain a backup copy during the iterative building process.<br /><br />Addition:<br /><br />In X11, which needs to be set to 800x600 8-bit mode, I had to uncomment these lines:<br /><pre> #Option "progLcdModeRegs" "true"<br /> #Option "progLcdModeStretch" "true"</pre><br />

May 08, 2014 07:35 PM

Grey Gandalf

DataBasin: advanced SObject describe

Grey Gandalf

I enhanced <a href="https://gap.nongnu.org/databasin/index.html">DataBasin</a> object's describe. First, I do parse the RecordTypeInfo now, but that was the easy part.&nbsp; Salesforce.com returns only the record type ID and name, but programmatically, this is not very useful. The important bit would be the Developer Name of each record type.<br /><br />I enhanced the results by querying automatically the RecordType table, extracting the Developer Name, matching it through the RT Id and merging back the results, so that the resulting DBSObject has a complete Record Type information, totally transparent.

May 08, 2014 01:50 PM

May 04, 2014

Grey Gandalf

ProjectCenter build parser fixes

Grey Gandalf

Debugging build tools...<br /><br />I was long plagued <a href="https://www.gnustep.org/">GNUstep</a>'s IDE <a href="https://www.gnustep.org/experience/ProjectCenter.html">ProjectCenter</a> had problems with parsing the compiler's output. This made "clicking" on the warning or error often impossible. I never dug into the details, but it happened more and more often and was worse on different systems than others.<br /><br />Yesterday, while working on another projects I got so annoyed that I debug until deep into the night to fix them and found two different sources:<br /><ul><li>On all OS recent versions of gcc (gcc 4.x) have a "fatal error" that was not being correctly parsed and assimilated to an error </li><li>OpenBSD was using egcc and this was causing the build files not to be recognized, so the error was parsed, but opening the correct editor was impossible</li></ul>I fixed both! Using modern gcc is thus now fine as using OpenBSD that works now like Linux. GNUstep's cross-platform support thus continues!<br /><br />Happy hacking.

May 04, 2014 02:45 PM

May 02, 2014

Grey Gandalf

DataBasin 0.7

Grey Gandalf

<a href="https://gap.nongnu.org/databasin/index.html">DataBasin</a> 0.7 is out!<br /><br />If you need to query, update,&nbsp; create data on SalesForce.com and are on a GNU/Linux, *BSD, Solaris system or MacOS-X... try it out! Do you need to perform a query having objects or IDs instead of a where clause? Select-identify will come handy for you.<br /><br />Some of the news:<br /><ul><li>Save results of update and create in a results.csv file with succeess and failure</li><li>Progress monitor of operations shows time remaining&nbsp;</li><li>The login panel shows a visual icon if login happened successfully or not</li><li>Progress monitor for Query, Delete, Insert</li><li>Select identify now supports ORDER BY</li><li>Internal core enhanced for Delete</li><li>Many bug fixes and clean-ups </li></ul>

May 02, 2014 10:11 AM

April 03, 2014

Grey Gandalf

Graphs: Improved Grid and labels

Grey Gandalf

Lots of new stuff in <a href="https://gap.nongnu.org/oresmekit/index.html">OresmeKit</a>, the graphing toolkit for GNUstep and Mac! As an Example, an advanced dashboard based on DataBasin that displays the system load of Salesforce.com. It is not generally available yet, but I hope it will be!<br /><br />Grid-sizing is now selectable so it gets spaced in 1K or 1M intervals (depending on the data-range available), like it is used in both screenshots in this example.<br /><br />&nbsp;<span>- (void)setYAxisGridSizing:(OKGridSizing)sizing;</span><br /><br />Can take now: OKGridConstantSize, OKGridKiloMega<br /><br />Also, one can decide to draw Just the label of the minimum and maximum value or a label for every grid:<br /><br /><span>- (void)setYAxisLabelStyle:(OKLabelStyle)style;</span><br /><br />Can take:&nbsp; OKNoLabels, OKMinMaxLabels, OKAllLabels<br /><br /><div class="separator"></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container"><tbody><tr><td><a href="https://4.bp.blogspot.com/-cKtL2Vsu3Hc/Uz14RJBH1AI/AAAAAAAAA5Y/_JPqKrGF2Fg/s1600/batch_tracker_col.png"><img border="0" src="https://4.bp.blogspot.com/-cKtL2Vsu3Hc/Uz14RJBH1AI/AAAAAAAAA5Y/_JPqKrGF2Fg/s1600/batch_tracker_col.png" height="320" width="308" /></a></td></tr><tr><td class="tr-caption">1000-unit Grid</td></tr></tbody></table><br /><div class="separator"></div><div class="separator"><br /></div><div class="separator"><br /></div><div class="separator">To complement this kind of visualization, a new kind of Label formatting can be used. In the example above, the numbers are plain, 10.000 is written as such, in the example below, it is formatted as 10K, if we were using 10.000.000, it would me 10M</div><div class="separator"><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container"><tbody><tr><td><a href="https://3.bp.blogspot.com/-YX-zync9Cts/Uz1lC-8aFmI/AAAAAAAAA4o/4TAnNeOmQh8/s1600/future_traker_Kdisplay.png"><img border="0" src="https://3.bp.blogspot.com/-YX-zync9Cts/Uz1lC-8aFmI/AAAAAAAAA4o/4TAnNeOmQh8/s1600/future_traker_Kdisplay.png" height="307" width="320" /></a></td></tr><tr><td class="tr-caption">1000 - grid with K formatting</td></tr></tbody></table><br />

April 03, 2014 06:03 PM

Subscriptions