From david.thompson at kitware.com Mon Jul 3 09:45:52 2017 From: david.thompson at kitware.com (David Thompson) Date: Mon, 3 Jul 2017 09:45:52 -0400 Subject: [Smtk-developers] Harmonizing ParaView and SMTK Views Message-ID: <84C59AD3-28BC-4A8A-ABF4-96FCEAA32A40@kitware.com> Hi all, I've been examining the issues involved with integrating SMTK views into ModelBuilder (and thus ParaView). Overall, PV views and SMTK views are not incompatible, but do have some different assumptions. * PV views expect most data to live on the server, while SMTK's current views (the attribute view and its specialized child that presents operator attributes) live on the client. * SMTK's Qt extension expects to manage a single attribute-view widget per qtUIManager. It's unclear to me whether creating multiple qtUIManagers attached to the same attribute system would accomplish. * We could add a new SMTK view type for PV render windows, but that could make maintaining ModelBuilder more complicated because changes to PV's render view might then require SMTK changes. I have a prototype showing some progress (screenshot attached; branch on gitlab.kitware.com/dcthomp/{smtk,cmb}.git, named attribute-view in both repos). What these branches do is 1. enable multiple views by not overwriting qtCMBMainWindow.ui's central widget. 2. add a new PV behavior that exposes PV's empty-view type-selector without also adding point/cell-selection buttons to the render view (which we don't want by default because CMB is in block-selection mode by default). 3. create a new subclass of ParaView's pqView named pqAttributeView (pqSMTKView would be more accurate); a server-side view object (an instance of vtkSMTKAttributeView) associated with the proxy; and a dummy, XML-only representation for this view. Unlike other views in PV, this view does not track the active dataset; instead it is passed a qtUIManager (more on this below) whose top-level SMTK-view it displays on the client. 4. configure pqAttributeView to emit a signal when it is initialized; ModelBuilder connects this signal to SimBuilder so that when a new template is loaded every existing pqAttributeView is informed of the new qtUIManager. The branch does *not* 1. incorporate all of John and Haocheng's multi-view work (yet). 2. disable the dockable "Attribute" panel. The issues that need to be resolved are: 1. A qtUIManager instance (such as the one created by SimBuilder when loading an attribute) can have only a single QWidget view, but ParaView allows multiple views of the same type. So, right now if you create 2 SMTK attribute views, all but the most-recently-created one will be a blank QWidget. 2. The signal connecting SimBuilder to pqAttributeView instances assumes that new pqAttributeViews should always display the currently-loaded simulation -- but another use case would be to display the active model's operator view, or more generally to allow views of multiple attribute systems at a time. Yet another use case would be to allow views to focus on a particular attribute in an attribute system (e.g., a material view, a BC view, an IC view, a solver-parameter view). In this latter use-case, there would not be a single, top-level view in an attribute file, which goes against the current convention. 3. Either we need to expand SMTK's ViewInfo objects to allow a view to be a render-view of model-entities or change the PV behavior mentioned above to be more explicit about what types of PV views we allow. 4. There's still not a clear path for items in a pqAttributeView's attribute to employ server-side PV widgets. The difficulty is that PV uses a dock-panel (the property inspector) to enable/disable widgets, which are only shown for the active pipeline object. However, in ModelBuilder, items in pqAttributeView which might be associated to 3D widgets would display those widgets in a different PV view (the render view). If we change visibility of a 3-D widget in the attribute view, which render-view would that affect? David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2017-07-01 at 23.11.59.png Type: image/png Size: 523319 bytes Desc: not available URL: From david.thompson at kitware.com Mon Jul 3 22:22:06 2017 From: david.thompson at kitware.com (David Thompson) Date: Mon, 3 Jul 2017 22:22:06 -0400 Subject: [Smtk-developers] CGM on Sierra Message-ID: <2D95F769-CE0A-4665-8549-8AE03DC9F43B@kitware.com> Hi Bob, I tried a superbuild with CGM enabled and ran into the same issue as you (OpenCascade, not CGM, fails to build with a time-macro conflict). It is easily fixed (and I wanna say I've done it before): change line 67 of superbuild/oce/src/src/OSD/gettime_osx.h from: #ifdef __APPLE__ to #if defined(__APPLE__) && !defined(CLOCK_REALTIME) I suspect that if we weren't using a really stale version of OpenCascade things would probably work. But I don't know whether CGM will work with a newer version of OpenCascade. David From ben.boeckel at kitware.com Wed Jul 5 13:53:21 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Wed, 5 Jul 2017 13:53:21 -0400 Subject: [Smtk-developers] Unhappy dashboards In-Reply-To: References: <688FE8C3-7870-41EC-B584-CC1F9BD89B2B@kitware.com> <27978598-EDEF-49CB-8E72-9114240D6AE5@kitware.com> <9EAE2867-6C8A-4CB9-A137-8190C926FC5B@kitware.com> Message-ID: <20170705175321.GC10435@megas.kitware.com> On Tue, May 30, 2017 at 11:00:39 -0400, David Thompson wrote: > I take that back. Apparently, the buildslave page does not report > pending builds and there are pending builds requested (at least > https://buildbot.kitware.com/builders/smtk-junction-osx-shared-release%2Bpybind11 > ). This is because builders have pending builds. Builders can go to many buildslaves which is why buildslaves don't list pending builds (we set it up so that there is only one available slave per builder, but that is not a "normal" use case). > So, if you log in to junction, what does "ps ax | grep buildslave" pgrep ;) . > say? Is it a zombie process or just not started? Do you know if Ben > was using launchctl to start the buildslave? All macOS machines use launchctl. Bob's run them as system daemons since they have other users log in at times whereas all the others are dedicated buildbots and run it as a daemon. --Ben From tj.corona at kitware.com Wed Jul 5 13:58:26 2017 From: tj.corona at kitware.com (TJ Corona) Date: Wed, 5 Jul 2017 13:58:26 -0400 Subject: [Smtk-developers] Unhappy dashboards In-Reply-To: <20170705175321.GC10435@megas.kitware.com> References: <688FE8C3-7870-41EC-B584-CC1F9BD89B2B@kitware.com> <27978598-EDEF-49CB-8E72-9114240D6AE5@kitware.com> <9EAE2867-6C8A-4CB9-A137-8190C926FC5B@kitware.com> <20170705175321.GC10435@megas.kitware.com> Message-ID: <73951797-72EA-456E-906C-01CE87968B98@kitware.com> Hey Ben, The issues you are reading about occurred a while ago, and I think they are no longer issues. The current state of the dashboards is ok, except the open.cdash CMB needs to have its list of responding machines updated (the names of the builds have changed, but I didn?t have permission to fix the cdash page). Sincerely, T.J. Thomas J. Corona, Ph.D. Kitware, Inc. Senior R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4443 > On Jul 5, 2017, at 1:53 PM, Ben Boeckel wrote: > > On Tue, May 30, 2017 at 11:00:39 -0400, David Thompson wrote: >> I take that back. Apparently, the buildslave page does not report >> pending builds and there are pending builds requested (at least >> https://buildbot.kitware.com/builders/smtk-junction-osx-shared-release%2Bpybind11 >> ). > > This is because builders have pending builds. Builders can go to many > buildslaves which is why buildslaves don't list pending builds (we set > it up so that there is only one available slave per builder, but that is > not a "normal" use case). > >> So, if you log in to junction, what does "ps ax | grep buildslave" > > pgrep ;) . > >> say? Is it a zombie process or just not started? Do you know if Ben >> was using launchctl to start the buildslave? > > All macOS machines use launchctl. Bob's run them as system daemons since > they have other users log in at times whereas all the others are > dedicated buildbots and run it as a daemon. > > --Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.thompson at kitware.com Thu Jul 6 10:52:55 2017 From: david.thompson at kitware.com (David Thompson) Date: Thu, 6 Jul 2017 10:52:55 -0400 Subject: [Smtk-developers] ParaView/CMB changes for multiple views Message-ID: Hi all, As I work on changes for CMB v5, one of the big features will be for ModelBuilder to allow multiple render-views instead of its current practice of creating a single default view that cannot be split or changed. I am thinking about using this opportunity to refactor the ModelBuilder code where it was confusing and would appreciate your input. I'll outline my plan below: Currently, CMB is organized as shown in the attached diagram: the pqCMBModelManager (which owns the smtk::model::Manager indirectly) creates and manages 1 pqSMTKModelInfo per model plus any auxiliary-geometry representations that are intended to be separate from the model. The pqSMTKModelInfo owns a pqSMTKMeshInfo instance (which owns multiple representations for the model's mesh collection) and several representations (1 for each separate auxiliary geometry image). I believe that a lot of the code can be simplified by removing the pqSMTK*Info and smtkAuxGeoInfo classes (none of which *were* representations but which owned representations) in favor of a new pqSMTKModelRepresentation class. I have some more due diligence to do: + it may require a subclass of vtkPVDataRepresentation on the server, but we may also be able to get by with just managing multiple proxies to existing server-side representations in pqSMTKModelRepresentation. + I'm not familiar with the interaction of selection and representations and need to ensure we'll be able to continue selecting things in a filtered way. If you can think of other problems with this or have better ideas, please let me know. David -------------- next part -------------- A non-text attachment was scrubbed... Name: cmb-v4-representations.pdf Type: application/pdf Size: 118552 bytes Desc: not available URL: From haocheng.liu at kitware.com Thu Jul 6 11:57:34 2017 From: haocheng.liu at kitware.com (Haocheng Liu) Date: Thu, 6 Jul 2017 11:57:34 -0400 Subject: [Smtk-developers] ParaView/CMB changes for multiple views In-Reply-To: References: Message-ID: Hi David, On Thu, Jul 6, 2017 at 10:52 AM, David Thompson wrote: > Hi all, > > As I work on changes for CMB v5, one of the big features will be for > ModelBuilder to allow multiple render-views instead of its current practice > of creating a single default view that cannot be split or changed. > > I am thinking about using this opportunity to refactor the ModelBuilder > code where it was confusing and would appreciate your input. I'll outline > my plan below: > > Currently, CMB is organized as shown in the attached diagram: the > pqCMBModelManager (which owns the smtk::model::Manager indirectly) creates > and manages 1 pqSMTKModelInfo per model plus any auxiliary-geometry > representations that are intended to be separate from the model. The > pqSMTKModelInfo owns a pqSMTKMeshInfo instance (which owns multiple > representations for the model's mesh collection) and several > representations (1 for each separate auxiliary geometry image). > I didn't find pqDataRepresentation(instance-glyphs) in pqSMTKModelInfo, I guess it's still in WIP? pqDataREpresentation(model) and pqDataRepresentation(selection) actually point to the same pqDataRepresentation. Paraview has two sources for pqDataRepresentation, one for selection(We use a vtkSMProxy in pqSMTKModelInfo to control selection) and another for normal rendering. Currently in CMB if user makes a selection, we would turn off the selected entities' visibility first then enable the selection source. See pqSMTKModelPanel::selectEntityRepresentations() for detail. > > I believe that a lot of the code can be simplified by removing the > pqSMTK*Info and smtkAuxGeoInfo classes (none of which *were* > representations but which owned representations) in favor of a new > pqSMTKModelRepresentation class. I have some more due diligence to do: > > + it may require a subclass of vtkPVDataRepresentation on the server, but > we may also be able to get by with just managing multiple proxies to > existing server-side representations in pqSMTKModelRepresentation. > + I'm not familiar with the interaction of selection and representations > and need to ensure we'll be able to continue selecting things in a filtered > way. > See my comment above. > > If you can think of other problems with this or have better ideas, please > let me know. > > +1 for this cool idea. > David > > > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers > > -- Best regards Haocheng Haocheng LIU Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4421 <(518)%20881-4421> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.boeckel at kitware.com Mon Jul 10 09:58:22 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Mon, 10 Jul 2017 09:58:22 -0400 Subject: [Smtk-developers] (no subject) In-Reply-To: References: <20170516211340.GA3063@megas.kitware.com> Message-ID: <20170710135822.GA5870@megas.kitware.com> On Tue, May 30, 2017 at 12:35:34 -0400, Maxim Torgonskiy wrote: > I've created a docker image (debian stretch) for the cmb compilation > to be sure that the build is clean and the error is reproducible. > Could you please take a look at it? > The config is the following: Sorry, I was on vacation all of June and finally got to this email. > -DENABLE_moab:BOOL=ON This builds using the superbuild's copy of MOAB rather than the one inside of SMTK. I'm afraid that this is not really tested. We should either drop the superbuild's copy or update it so that it at least works. --Ben From bob.obara at kitware.com Tue Jul 11 10:46:00 2017 From: bob.obara at kitware.com (Bob Obara) Date: Tue, 11 Jul 2017 10:46:00 -0400 Subject: [Smtk-developers] Save vs Save As Message-ID: Hi All, I think the way we have defined Save vs Save As might have a couple issues. I?ve created a Wiki Page: https://gitlab.kitware.com/cmb/smtk/wikis/issues-with-smtk-save-and-save-as that describes the issue and possible ways to get around it. Please feel free to make changes and express opinions. Thanks! Bob Robert M. O'Bara, MEng. Assistant Director of Scientific Computing Kitware Inc. 28 Corporate Drive Suite 101 Clifton Park, NY 12065 Phone: (518) 881- 4931 -------------- next part -------------- An HTML attachment was scrubbed... URL: From tj.corona at kitware.com Tue Jul 11 19:24:58 2017 From: tj.corona at kitware.com (TJ Corona) Date: Tue, 11 Jul 2017 19:24:58 -0400 Subject: [Smtk-developers] TalosIV is offline again Message-ID: <6A9C35ED-CD4E-44BC-A69B-5D4883B500C5@kitware.com> If anyone is still at the office, TalosIV is offline again. This means we will have to manually post ES packages again tomorrow. Thomas J. Corona, Ph.D. Kitware, Inc. Senior R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4443 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.boeckel at kitware.com Wed Jul 12 09:16:46 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Wed, 12 Jul 2017 09:16:46 -0400 Subject: [Smtk-developers] TalosIV is offline again In-Reply-To: <6A9C35ED-CD4E-44BC-A69B-5D4883B500C5@kitware.com> References: <6A9C35ED-CD4E-44BC-A69B-5D4883B500C5@kitware.com> Message-ID: <20170712131646.GB17906@megas.kitware.com> On Tue, Jul 11, 2017 at 19:24:58 -0400, TJ Corona wrote: > If anyone is still at the office, TalosIV is offline again. This means > we will have to manually post ES packages again tomorrow. It built packages. It being offline in buildbot is not related to it not being powered on. --Ben From david.thompson at kitware.com Sat Jul 15 16:09:01 2017 From: david.thompson at kitware.com (David Thompson) Date: Sat, 15 Jul 2017 16:09:01 -0400 Subject: [Smtk-developers] Qt5 build Message-ID: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> Hi all, Has anyone got SMTK building with Qt5 and SMTK_ENABLE_CUMULUS turned on? I've hit a few errors so far: + No QT5_COMPONENTS is specified in the CMakeLists; FindQt5.cmake complains at least one component is required. + jobtablewidget.cxx:44:34: error: no member named 'setResizeMode' in 'QHeaderView'; did you mean 'sectionResizeMode'? + smtk/extension/cumulus/cumulusproxy.cxx fails to build because its header does not appear to include but the class inherits QObject. Am I doing something wrong or does smtkCumulusExt need to be updated for Qt5? Thanks, David From tj.corona at kitware.com Sun Jul 16 21:17:45 2017 From: tj.corona at kitware.com (TJ Corona) Date: Sun, 16 Jul 2017 21:17:45 -0400 Subject: [Smtk-developers] Another crazy idea Message-ID: <93437694-3710-46C8-B424-B1ABC50BFAD2@kitware.com> Hi all, I?ve been thinking about ways to make smtk/cmb more accessible to a novice user (one of the things that may help to grow a larger community of non-Kitware developers). Teeing off of Alvaro?s GUI .sbt creator, I was thinking that we could create a GUI creator for making operators. It would essentially be the .sbt creator, but it would also generate a skeleton .py (or .cxx) file that would have generated code for accessing the inputs and populating the outputs that were chosen by the user in the .sbt file. We could pair this with a means of loading .py operators in ModelBuilder at runtime (which would basically a) load the .py module and b) copy the input .py file to a users directory that is loaded at startup). This would lower the bar for creating operators to just understanding how to fill in the ?guts? of an operator. I know Amanda at ERDC has begun learning how to write operators, and something like this could really simplify that process. All the pieces seem to be already in place, so the only thing we would need is the motivation to put it together. Do you guys think something like this could be useful enough to invest some development time? Sincerely, T.J. Thomas J. Corona, Ph.D. Kitware, Inc. Senior R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4443 -------------- next part -------------- An HTML attachment was scrubbed... URL: From haocheng.liu at kitware.com Mon Jul 17 09:37:00 2017 From: haocheng.liu at kitware.com (Haocheng Liu) Date: Mon, 17 Jul 2017 09:37:00 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> Message-ID: I would give it a shot today. On Sat, Jul 15, 2017 at 4:09 PM, David Thompson wrote: > Hi all, > > Has anyone got SMTK building with Qt5 and SMTK_ENABLE_CUMULUS turned on? > I've hit a few errors so far: > > + No QT5_COMPONENTS is specified in the CMakeLists; FindQt5.cmake > complains at least one component is required. > + jobtablewidget.cxx:44:34: error: no member named 'setResizeMode' in > 'QHeaderView'; did you mean 'sectionResizeMode'? > + smtk/extension/cumulus/cumulusproxy.cxx fails to build because its > header does not appear to include but the class inherits QObject. > > Am I doing something wrong or does smtkCumulusExt need to be updated for > Qt5? > > Thanks, > David > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers > -- Best regards Haocheng Haocheng LIU Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4421 <(518)%20881-4421> -------------- next part -------------- An HTML attachment was scrubbed... URL: From haocheng.liu at kitware.com Mon Jul 17 10:05:09 2017 From: haocheng.liu at kitware.com (Haocheng Liu) Date: Mon, 17 Jul 2017 10:05:09 -0400 Subject: [Smtk-developers] Another crazy idea In-Reply-To: <93437694-3710-46C8-B424-B1ABC50BFAD2@kitware.com> References: <93437694-3710-46C8-B424-B1ABC50BFAD2@kitware.com> Message-ID: Hi TJ, I think it's a really cool idea, it would greatly reduce the learning curve of understanding how SMTK read inputs(attribute system)and processes operation result(mesh_expunged, modified...etc). On Sun, Jul 16, 2017 at 9:17 PM, TJ Corona wrote: > Hi all, > > I?ve been thinking about ways to make smtk/cmb more accessible to a > novice user (one of the things that may help to grow a larger community of > non-Kitware developers). Teeing off of Alvaro?s GUI .sbt creator, I was > thinking that we could create a GUI creator for making operators. It would > essentially be the .sbt creator, but it would also generate a skeleton .py > (or .cxx) file that would have generated code for accessing the inputs and > populating the outputs that were chosen by the user in the .sbt file. We > could pair this with a means of loading .py operators in ModelBuilder at > runtime (which would basically a) load the .py module and b) copy the > input .py file to a users directory that is loaded at startup). This would > lower the bar for creating operators to just understanding how to fill in > the ?guts? of an operator. > >From my experience, a serious gap for creating opeator is to understand how to get the input from specification and add entities to the result. Once user has the head and tail of the operator, all user needs to do is to create the body. I would like to generate these infos for them in the skeleton .py(or .cxx). I know Amanda at ERDC has begun learning how to write operators, and > something like this could really simplify that process. All the pieces seem > to be already in place, so the only thing we would need is the motivation > to put it together. > + 1. > Do you guys think something like this could be useful enough to invest > some development time? > > Sincerely, > T.J. > > Operator creator might should contain these four parts: - session (I know discrete session and polygon session have their own logic of transcribing data. So we should warning the developer before they shoot their foot) - input for the operator - output for the operator - .cxx /.py Thomas J. Corona, Ph.D. > Kitware, Inc. > Senior R&D Engineer > 21 Corporate Drive > Clifton Park, NY 12065-8662 > Phone: 518-881-4443 <(518)%20881-4443> > > > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers > > -- Best regards Haocheng Haocheng LIU Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4421 <(518)%20881-4421> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.obara at kitware.com Mon Jul 17 10:17:45 2017 From: bob.obara at kitware.com (Bob Obara) Date: Mon, 17 Jul 2017 10:17:45 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> Message-ID: <9A9EF8A2-19F0-4920-8780-24B91C85E89E@kitware.com> I?ve done it without the Cumulus part. Where are you pointing your QT5 variable? Bob Robert M. O'Bara, MEng. Assistant Director of Scientific Computing Kitware Inc. 28 Corporate Drive Suite 101 Clifton Park, NY 12065 Phone: (518) 881- 4931 > On Jul 15, 2017, at 4:09 PMEDT, David Thompson wrote: > > Hi all, > > Has anyone got SMTK building with Qt5 and SMTK_ENABLE_CUMULUS turned on? I've hit a few errors so far: > > + No QT5_COMPONENTS is specified in the CMakeLists; FindQt5.cmake complains at least one component is required. > + jobtablewidget.cxx:44:34: error: no member named 'setResizeMode' in 'QHeaderView'; did you mean 'sectionResizeMode'? > + smtk/extension/cumulus/cumulusproxy.cxx fails to build because its header does not appear to include but the class inherits QObject. > > Am I doing something wrong or does smtkCumulusExt need to be updated for Qt5? > > Thanks, > David > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.thompson at kitware.com Mon Jul 17 10:24:31 2017 From: david.thompson at kitware.com (David Thompson) Date: Mon, 17 Jul 2017 10:24:31 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: <9A9EF8A2-19F0-4920-8780-24B91C85E89E@kitware.com> References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> <9A9EF8A2-19F0-4920-8780-24B91C85E89E@kitware.com> Message-ID: Hi Bob, > I?ve done it without the Cumulus part. > Where are you pointing your QT5 variable? To a homebrew Qt 5.9.1 build (/usr/local/opt/qt/lib/cmake/Qt5/Qt5Config.cmake). David From david.thompson at kitware.com Mon Jul 17 10:48:41 2017 From: david.thompson at kitware.com (David Thompson) Date: Mon, 17 Jul 2017 10:48:41 -0400 Subject: [Smtk-developers] Another crazy idea In-Reply-To: <93437694-3710-46C8-B424-B1ABC50BFAD2@kitware.com> References: <93437694-3710-46C8-B424-B1ABC50BFAD2@kitware.com> Message-ID: Hi TJ, > I?ve been thinking about ways to make smtk/cmb more accessible ... I was thinking that we could create a GUI creator for making operators. ... All the pieces seem to be already in place, so the only thing we would need is the motivation to put it together. Do you guys think something like this could be useful enough to invest some development time? I think that would be great, especially if we add a tutorial for it (or modify the existing tutorial). David From bob.obara at kitware.com Mon Jul 17 10:53:31 2017 From: bob.obara at kitware.com (Bob Obara) Date: Mon, 17 Jul 2017 10:53:31 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> <9A9EF8A2-19F0-4920-8780-24B91C85E89E@kitware.com> Message-ID: <78C51E20-4CC7-4404-811A-470C90B63428@kitware.com> Thats what I have - does it build when Cumulus support is turned off? Bob? Robert M. O'Bara, MEng. Assistant Director of Scientific Computing Kitware Inc. 28 Corporate Drive Suite 101 Clifton Park, NY 12065 Phone: (518) 881- 4931 > On Jul 17, 2017, at 10:24 AMEDT, David Thompson wrote: > > Hi Bob, > >> I?ve done it without the Cumulus part. >> Where are you pointing your QT5 variable? > > To a homebrew Qt 5.9.1 build (/usr/local/opt/qt/lib/cmake/Qt5/Qt5Config.cmake). > > David -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.thompson at kitware.com Mon Jul 17 11:02:30 2017 From: david.thompson at kitware.com (David Thompson) Date: Mon, 17 Jul 2017 11:02:30 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: <78C51E20-4CC7-4404-811A-470C90B63428@kitware.com> References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> <9A9EF8A2-19F0-4920-8780-24B91C85E89E@kitware.com> <78C51E20-4CC7-4404-811A-470C90B63428@kitware.com> Message-ID: <23F10646-5090-4716-B5FB-7A80AED591A5@kitware.com> > Thats what I have - does it build when Cumulus support is turned off? Yes. David From haocheng.liu at kitware.com Mon Jul 17 11:29:07 2017 From: haocheng.liu at kitware.com (Haocheng Liu) Date: Mon, 17 Jul 2017 11:29:07 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> Message-ID: On Sat, Jul 15, 2017 at 4:09 PM, David Thompson wrote: > Hi all, > > Has anyone got SMTK building with Qt5 and SMTK_ENABLE_CUMULUS turned on? > I've hit a few errors so far: > > Nope... It fails on Linux as well. Cumulus needs love for qt5. I am looking into it right now. > + No QT5_COMPONENTS is specified in the CMakeLists; FindQt5.cmake > complains at least one component is required. > + jobtablewidget.cxx:44:34: error: no member named 'setResizeMode' in > 'QHeaderView'; did you mean 'sectionResizeMode'? > Yep. Qt5 Changed the API of QHeaderView. > + smtk/extension/cumulus/cumulusproxy.cxx fails to build because its > header does not appear to include but the class inherits QObject. > > Am I doing something wrong or does smtkCumulusExt need to be updated for > Qt5? > > Thanks, > David > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers > -- Best regards Haocheng Haocheng LIU Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4421 <(518)%20881-4421> -------------- next part -------------- An HTML attachment was scrubbed... URL: From tj.corona at kitware.com Mon Jul 17 12:40:03 2017 From: tj.corona at kitware.com (TJ Corona) Date: Mon, 17 Jul 2017 12:40:03 -0400 Subject: [Smtk-developers] Another crazy idea In-Reply-To: References: <93437694-3710-46C8-B424-B1ABC50BFAD2@kitware.com> Message-ID: <695A125F-47AE-4C3C-9A8D-D186ABFA123E@kitware.com> Ah, speaking of which, I should definitely update the existing tutorial for the python operator stuff. Thanks for reminding me! Thomas J. Corona, Ph.D. Kitware, Inc. Senior R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4443 > On Jul 17, 2017, at 10:48 AM, David Thompson wrote: > > Hi TJ, > >> I?ve been thinking about ways to make smtk/cmb more accessible ... I was thinking that we could create a GUI creator for making operators. ... All the pieces seem to be already in place, so the only thing we would need is the motivation to put it together. Do you guys think something like this could be useful enough to invest some development time? > > I think that would be great, especially if we add a tutorial for it (or modify the existing tutorial). > > David -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.thompson at kitware.com Mon Jul 17 13:33:19 2017 From: david.thompson at kitware.com (David Thompson) Date: Mon, 17 Jul 2017 13:33:19 -0400 Subject: [Smtk-developers] Another crazy idea In-Reply-To: <695A125F-47AE-4C3C-9A8D-D186ABFA123E@kitware.com> References: <93437694-3710-46C8-B424-B1ABC50BFAD2@kitware.com> <695A125F-47AE-4C3C-9A8D-D186ABFA123E@kitware.com> Message-ID: <522CC07D-05FB-4649-A829-7A9C236B454B@kitware.com> Hi TJ, > Ah, speaking of which, I should definitely update the existing tutorial for the python operator stuff. ... It doesn't touch the tutorials, but note that the smtk!703 MR adds some reference documentation for operators (what the different created/modified/expunged/tess_changed items are for, etc.). https://gitlab.kitware.com/cmb/smtk/merge_requests/703/diffs#944eab6b03f3550a0502078cacc240654f5b7afb_0_1 David From haocheng.liu at kitware.com Mon Jul 17 13:54:12 2017 From: haocheng.liu at kitware.com (Haocheng Liu) Date: Mon, 17 Jul 2017 13:54:12 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> Message-ID: Hi David, Would you please try the following smtk patch? It should iron out most of the errors. For my trial, the only problem is about how to build qt5 with openssl enabled on Linux... A homebrew qt5 on mac should be able to bypass the error as "qt5 cannot find openssl". On Mon, Jul 17, 2017 at 11:29 AM, Haocheng Liu wrote: > > > On Sat, Jul 15, 2017 at 4:09 PM, David Thompson < > david.thompson at kitware.com> wrote: > >> Hi all, >> >> Has anyone got SMTK building with Qt5 and SMTK_ENABLE_CUMULUS turned on? >> I've hit a few errors so far: >> >> Nope... It fails on Linux as well. Cumulus needs love for qt5. I am > looking into it right now. > >> + No QT5_COMPONENTS is specified in the CMakeLists; FindQt5.cmake >> complains at least one component is required. >> + jobtablewidget.cxx:44:34: error: no member named 'setResizeMode' in >> 'QHeaderView'; did you mean 'sectionResizeMode'? >> > Yep. Qt5 Changed the API of QHeaderView. > >> + smtk/extension/cumulus/cumulusproxy.cxx fails to build because its >> header does not appear to include but the class inherits QObject. >> >> Am I doing something wrong or does smtkCumulusExt need to be updated for >> Qt5? >> >> Thanks, >> David >> _______________________________________________ >> Smtk-developers mailing list >> Smtk-developers at smtk.org >> http://public.kitware.com/mailman/listinfo/smtk-developers >> > > > > -- > Best regards > Haocheng > > Haocheng LIU > Kitware, Inc. > R&D Engineer > 21 Corporate Drive > Clifton Park, NY 12065-8662 > Phone: 518-881-4421 <(518)%20881-4421> > -- Best regards Haocheng Haocheng LIU Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4421 <(518)%20881-4421> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smtk.patch Type: text/x-patch Size: 4405 bytes Desc: not available URL: From bob.obara at kitware.com Mon Jul 17 13:59:53 2017 From: bob.obara at kitware.com (Bob Obara) Date: Mon, 17 Jul 2017 13:59:53 -0400 Subject: [Smtk-developers] CGM on Sierra In-Reply-To: <2D95F769-CE0A-4665-8549-8AE03DC9F43B@kitware.com> References: <2D95F769-CE0A-4665-8549-8AE03DC9F43B@kitware.com> Message-ID: Hey Ben, Can you please patch our cgm? Thanks! Bob Robert M. O'Bara, MEng. Assistant Director of Scientific Computing Kitware Inc. 28 Corporate Drive Suite 101 Clifton Park, NY 12065 Phone: (518) 881- 4931 > On Jul 3, 2017, at 10:22 PMEDT, David Thompson wrote: > > Hi Bob, > > I tried a superbuild with CGM enabled and ran into the same issue as you (OpenCascade, not CGM, fails to build with a time-macro conflict). It is easily fixed (and I wanna say I've done it before): change line 67 of superbuild/oce/src/src/OSD/gettime_osx.h from: > > #ifdef __APPLE__ > > to > > #if defined(__APPLE__) && !defined(CLOCK_REALTIME) > > I suspect that if we weren't using a really stale version of OpenCascade things would probably work. But I don't know whether CGM will work with a newer version of OpenCascade. > > David -------------- next part -------------- An HTML attachment was scrubbed... URL: From tj.corona at kitware.com Mon Jul 17 14:28:30 2017 From: tj.corona at kitware.com (TJ Corona) Date: Mon, 17 Jul 2017 14:28:30 -0400 Subject: [Smtk-developers] CGM on Sierra In-Reply-To: References: <2D95F769-CE0A-4665-8549-8AE03DC9F43B@kitware.com> Message-ID: <7CB77376-57B9-419C-93A3-C7940DA9EFAA@kitware.com> You can also get CGM to build if you pull from the latest master. That breaks a few things in the CGM session, though. Thomas J. Corona, Ph.D. Kitware, Inc. Senior R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4443 > On Jul 17, 2017, at 1:59 PM, Bob Obara wrote: > > Hey Ben, > > Can you please patch our cgm? > > Thanks! > > Bob > > Robert M. O'Bara, MEng. > Assistant Director of Scientific Computing > > Kitware Inc. > 28 Corporate Drive > Suite 101 > Clifton Park, NY 12065 > > Phone: (518) 881- 4931 > > > > >> On Jul 3, 2017, at 10:22 PMEDT, David Thompson > wrote: >> >> Hi Bob, >> >> I tried a superbuild with CGM enabled and ran into the same issue as you (OpenCascade, not CGM, fails to build with a time-macro conflict). It is easily fixed (and I wanna say I've done it before): change line 67 of superbuild/oce/src/src/OSD/gettime_osx.h from: >> >> #ifdef __APPLE__ >> >> to >> >> #if defined(__APPLE__) && !defined(CLOCK_REALTIME) >> >> I suspect that if we weren't using a really stale version of OpenCascade things would probably work. But I don't know whether CGM will work with a newer version of OpenCascade. >> >> David > > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.thompson at kitware.com Mon Jul 17 16:52:58 2017 From: david.thompson at kitware.com (David Thompson) Date: Mon, 17 Jul 2017 16:52:58 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> Message-ID: Hi Haocheng, > Would you please try the following smtk patch? That works, although I do get some warnings (pasted below). > It should iron out most of the errors. For my trial, the only problem is about how to build qt5 with openssl enabled on Linux... A homebrew qt5 on mac should be able to bypass the error as "qt5 cannot find openssl". I did not see any SSL errors/warnings using homebrew's qt 5.9.1. David [16/68] Building CXX object ThirdParty/SMTK/smtk/extension/cumulus/CMakeFiles/smtkCumulusExt.dir/cumulusproxy.cxx.o /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/cumulusproxy.cxx:158:54: warning: unused parameter 'newtSessionId' [-Wunused-parameter] void CumulusProxy::authenticateGirder(const QString& newtSessionId) ^ /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/cumulusproxy.cxx:355:76: warning: unused parameter 'errors' [-Wunused-parameter] void CumulusProxy::sslErrors(QNetworkReply* reply, const QList& errors) ^ 2 warnings generated. [17/68] Building CXX object ThirdParty/SMTK/smtk/extension/cumulus/CMakeFiles/smtkCumulusExt.dir/girderrequest.cxx.o /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/girderrequest.cxx:315:21: warning: unused variable 'request' [-Wunused-variable] ListItemsRequest* request = qobject_cast(this->sender()); ^ /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/girderrequest.cxx:348:23: warning: unused variable 'request' [-Wunused-variable] ListFoldersRequest* request = qobject_cast(this->sender()); ^ /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/girderrequest.cxx:420:21: warning: unused variable 'request' [-Wunused-variable] ListItemsRequest* request = qobject_cast(this->sender()); ^ 3 warnings generated. [18/68] Building CXX object ThirdParty/SMTK/smtk/extension/cumulus/CMakeFiles/smtkCumulusExt.dir/cumuluswidget.cxx.o /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/cumuluswidget.cxx:28:5: warning: field 'm_loginDialog' will be initialized after field 'm_jobTableModel' [-Wreorder] , m_loginDialog(this) ^ 1 warning generated. From haocheng.liu at kitware.com Mon Jul 17 17:33:08 2017 From: haocheng.liu at kitware.com (Haocheng Liu) Date: Mon, 17 Jul 2017 17:33:08 -0400 Subject: [Smtk-developers] Qt5 build In-Reply-To: References: <21B8601B-13A6-438C-8385-7E4AF4778BEE@kitware.com> Message-ID: On Mon, Jul 17, 2017 at 4:52 PM, David Thompson wrote: > Hi Haocheng, > > > Would you please try the following smtk patch? > > That works, although I do get some warnings (pasted below). > > > It should iron out most of the errors. For my trial, the only problem is > about how to build qt5 with openssl enabled on Linux... A homebrew qt5 on > mac should be able to bypass the error as "qt5 cannot find openssl". > > I did not see any SSL errors/warnings using homebrew's qt 5.9.1. > > David > > > [16/68] Building CXX object ThirdParty/SMTK/smtk/ > extension/cumulus/CMakeFiles/smtkCumulusExt.dir/cumulusproxy.cxx.o > /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/cumulusproxy.cxx:158:54: > warning: unused parameter 'newtSessionId' [-Wunused-parameter] > void CumulusProxy::authenticateGirder(const QString& newtSessionId) > ^ > /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/cumulusproxy.cxx:355:76: > warning: unused parameter 'errors' [-Wunused-parameter] > void CumulusProxy::sslErrors(QNetworkReply* reply, const > QList& errors) > > ^ > 2 warnings generated. > [17/68] Building CXX object ThirdParty/SMTK/smtk/ > extension/cumulus/CMakeFiles/smtkCumulusExt.dir/girderrequest.cxx.o > /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/girderrequest.cxx:315:21: > warning: unused variable 'request' [-Wunused-variable] > ListItemsRequest* request = qobject_cast >(this->sender()); > ^ > /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/girderrequest.cxx:348:23: > warning: unused variable 'request' [-Wunused-variable] > ListFoldersRequest* request = qobject_cast(this-> > sender()); > ^ > /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/girderrequest.cxx:420:21: > warning: unused variable 'request' [-Wunused-variable] > ListItemsRequest* request = qobject_cast >(this->sender()); > ^ > 3 warnings generated. > [18/68] Building CXX object ThirdParty/SMTK/smtk/ > extension/cumulus/CMakeFiles/smtkCumulusExt.dir/cumuluswidget.cxx.o > /stage/source/cmb/5/ThirdParty/SMTK/smtk/extension/cumulus/cumuluswidget.cxx:28:5: > warning: field 'm_loginDialog' will be initialized after field > 'm_jobTableModel' [-Wreorder] > , m_loginDialog(this) > ^ > 1 warning generated. > > Fixed. I would create a MR and ask Ben to review it. -- Best regards Haocheng Haocheng LIU Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4421 <(518)%20881-4421> -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.thompson at kitware.com Tue Jul 18 08:33:51 2017 From: david.thompson at kitware.com (David Thompson) Date: Tue, 18 Jul 2017 08:33:51 -0400 Subject: [Smtk-developers] Dashboards Message-ID: <33006408-30BE-4B43-A2DA-841A4530A8CA@kitware.com> Hi all, There appears to be some CMB and a little SMTK dashboard breakage. Most of it is caused by CMB builds using old versions of SMTK. However there are some other issues: 1. SMTK has some warnings due to a new ParaView/VTK that changes vtkArrayCalculator API: https://open.cdash.org/viewBuildError.php?type=1&buildid=4981646 The problem is that vtkVersionMacros.h cannot be used to eliminate this warning with development versions of VTK: development versions report the version number as 8.1.0, but not all commits with this version number have the API change present. 2. SMTK's junction, talosiv, and praxis builds report test failures like this: https://open.cdash.org/viewTest.php?onlyfailed&buildid=4981644 https://open.cdash.org/viewTest.php?onlyfailed&buildid=4982159 https://open.cdash.org/viewTest.php?onlyfailed&buildid=4981943 which are a result of dirty build trees (old libraries are being loaded, causing link issues). 3. Some SMTK and CMB builds appear to have problems because Qt5 is not configured: https://open.cdash.org/viewBuildError.php?buildid=4981865 https://open.cdash.org/viewBuildError.php?buildid=4981840 I am happy to help out with #1 but the others will take someone with access to the dashboard machines. David From ben.boeckel at kitware.com Fri Jul 21 16:28:46 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Fri, 21 Jul 2017 16:28:46 -0400 Subject: [Smtk-developers] git-lfs test repository Message-ID: <20170721202846.GA11628@rotor.kitware.com> Hi, I've pushed a repository to gitlab with CMB + testing data in LFS: https://gitlab.kitware.com/ben.boeckel/cmb-lfs-test Please fork, make changes, file MRs, and play around with it to see what it's like. git-lfs is available here: https://git-lfs.github.com/ and you'll need to run: git lfs install --local in order to enable the LFS bits. --Ben From david.thompson at kitware.com Sun Jul 23 17:05:42 2017 From: david.thompson at kitware.com (David Thompson) Date: Sun, 23 Jul 2017 17:05:42 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: <20170721202846.GA11628@rotor.kitware.com> References: <20170721202846.GA11628@rotor.kitware.com> Message-ID: <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> Hi Ben, > I've pushed a repository to gitlab with CMB + testing data in LFS: > > https://gitlab.kitware.com/ben.boeckel/cmb-lfs-test > > Please fork, make changes, file MRs, and play around with it to see what > it's like. I only see a single file (data/simpleadh.dat) being managed by git-lfs (run git lfs ls-files to see). The others look like they have been committed straight into the repo. Maybe the line in .gitattributes wasn't enough? David From ben.boeckel at kitware.com Mon Jul 24 10:58:54 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Mon, 24 Jul 2017 10:58:54 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> Message-ID: <20170724145854.GD23225@megas.kitware.com> On Sun, Jul 23, 2017 at 17:05:42 -0400, David Thompson wrote: > I only see a single file (data/simpleadh.dat) being managed by git-lfs > (run git lfs ls-files to see). The others look like they have been > committed straight into the repo. Maybe the line in .gitattributes > wasn't enough? Oops, forgot the recursive glob. Force-pushing a new branch. --Ben From ben.boeckel at kitware.com Mon Jul 24 11:50:05 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Mon, 24 Jul 2017 11:50:05 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: <20170724145854.GD23225@megas.kitware.com> References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> Message-ID: <20170724155005.GA2206@rotor.kitware.com> On Mon, Jul 24, 2017 at 10:58:54 -0400, Ben Boeckel wrote: > Oops, forgot the recursive glob. Force-pushing a new branch. Pushed. --Ben From david.thompson at kitware.com Tue Jul 25 11:09:27 2017 From: david.thompson at kitware.com (David Thompson) Date: Tue, 25 Jul 2017 11:09:27 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: <20170724155005.GA2206@rotor.kitware.com> References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> Message-ID: >> Oops, forgot the recursive glob. Force-pushing a new branch. > > Pushed. The previous version (only 1 file used git-lfs) cloned in 8 minutes. I am over 32 minutes into cloning the new version and barely half-way through (by total size). Even if newer versions of git/git-lfs are faster (I am using the macos-provided 2.11.0 and homebrew git-lfs 1.5.3), will it be enough? David From ben.boeckel at kitware.com Tue Jul 25 11:54:04 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Tue, 25 Jul 2017 11:54:04 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> Message-ID: <20170725155404.GA485@megas.kitware.com> On Tue, Jul 25, 2017 at 11:09:27 -0400, David Thompson wrote: > The previous version (only 1 file used git-lfs) cloned in 8 minutes. I > am over 32 minutes into cloning the new version and barely half-way > through (by total size). Even if newer versions of git/git-lfs are > faster (I am using the macos-provided 2.11.0 and homebrew git-lfs > 1.5.3), will it be enough? OK, this seems like a KHQ/KRS difference, nothing inherent. It cloned here in seconds (separate clone/fetch steps for the LFS data took ~30 seconds). Versions I'm using: git 2.9.4 git-lfs 1.5.5 Starting with 2.11.0, git can run filters through a single process (rather than a new process per filtered path). Note that there is also `git lfs clone` which defers content download until after the non-LFS data is downloaded. What's the performance of that? How about a clone without LFS at all and then followed by: git lfs fetch git lfs checkout ? --Ben From haocheng.liu at kitware.com Tue Jul 25 13:24:19 2017 From: haocheng.liu at kitware.com (Haocheng Liu) Date: Tue, 25 Jul 2017 13:24:19 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: <20170725155404.GA485@megas.kitware.com> References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> <20170725155404.GA485@megas.kitware.com> Message-ID: Using *git lfs clone* with git-lfs 2.2.1+ git 2.13, it takes me about 1 minute to download the data. With simple *git clone*, it feels like taking forever... On Tue, Jul 25, 2017 at 11:54 AM, Ben Boeckel wrote: > On Tue, Jul 25, 2017 at 11:09:27 -0400, David Thompson wrote: > > The previous version (only 1 file used git-lfs) cloned in 8 minutes. I > > am over 32 minutes into cloning the new version and barely half-way > > through (by total size). Even if newer versions of git/git-lfs are > > faster (I am using the macos-provided 2.11.0 and homebrew git-lfs > > 1.5.3), will it be enough? > > OK, this seems like a KHQ/KRS difference, nothing inherent. It cloned > here in seconds (separate clone/fetch steps for the LFS data took ~30 > seconds). > > Versions I'm using: > > git 2.9.4 > git-lfs 1.5.5 > > Starting with 2.11.0, git can run filters through a single process > (rather than a new process per filtered path). > > Note that there is also `git lfs clone` which defers content download > until after the non-LFS data is downloaded. What's the performance of > that? How about a clone without LFS at all and then followed by: > > git lfs fetch > git lfs checkout > > ? > > --Ben > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers > -- Best regards Haocheng Haocheng LIU Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4421 <(518)%20881-4421> -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcus.hanwell at kitware.com Tue Jul 25 13:43:43 2017 From: marcus.hanwell at kitware.com (Marcus D. Hanwell) Date: Tue, 25 Jul 2017 13:43:43 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> <20170725155404.GA485@megas.kitware.com> Message-ID: That is odd, I was interested, and just tried cloning $ time git clone https://gitlab.kitware.com/ben.boeckel/cmb-lfs-test.git real 0m39.754s user 0m7.284s sys 0m3.038s This is with git 2.13.3 $ time git lfs clone https://gitlab.kitware.com/ben.boeckel/cmb-lfs-test.git real 0m18.448s user 0m11.474s sys 0m3.975s Definitely faster, and easier to follow output, but both pretty fast from 21 on a desktop Linux machine. On Tue, Jul 25, 2017 at 1:24 PM, Haocheng Liu wrote: > Using *git lfs clone* with git-lfs 2.2.1+ git 2.13, it takes me about 1 > minute to download the data. With simple *git clone*, it feels like > taking forever... > > On Tue, Jul 25, 2017 at 11:54 AM, Ben Boeckel > wrote: > >> On Tue, Jul 25, 2017 at 11:09:27 -0400, David Thompson wrote: >> > The previous version (only 1 file used git-lfs) cloned in 8 minutes. I >> > am over 32 minutes into cloning the new version and barely half-way >> > through (by total size). Even if newer versions of git/git-lfs are >> > faster (I am using the macos-provided 2.11.0 and homebrew git-lfs >> > 1.5.3), will it be enough? >> >> OK, this seems like a KHQ/KRS difference, nothing inherent. It cloned >> here in seconds (separate clone/fetch steps for the LFS data took ~30 >> seconds). >> >> Versions I'm using: >> >> git 2.9.4 >> git-lfs 1.5.5 >> >> Starting with 2.11.0, git can run filters through a single process >> (rather than a new process per filtered path). >> >> Note that there is also `git lfs clone` which defers content download >> until after the non-LFS data is downloaded. What's the performance of >> that? How about a clone without LFS at all and then followed by: >> >> git lfs fetch >> git lfs checkout >> >> ? >> >> --Ben >> _______________________________________________ >> Smtk-developers mailing list >> Smtk-developers at smtk.org >> http://public.kitware.com/mailman/listinfo/smtk-developers >> > > > > -- > Best regards > Haocheng > > Haocheng LIU > Kitware, Inc. > R&D Engineer > 21 Corporate Drive > Clifton Park, NY 12065-8662 > Phone: 518-881-4421 <(518)%20881-4421> > > _______________________________________________ > Smtk-developers mailing list > Smtk-developers at smtk.org > http://public.kitware.com/mailman/listinfo/smtk-developers > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.thompson at kitware.com Tue Jul 25 15:32:10 2017 From: david.thompson at kitware.com (David Thompson) Date: Tue, 25 Jul 2017 15:32:10 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> <20170725155404.GA485@megas.kitware.com> Message-ID: Hi Marcus, > That is odd, I was interested, and just tried cloning > > $ time git clone https://gitlab.kitware.com/ben.boeckel/cmb-lfs-test.git > real 0m39.754s > $ time git lfs clone https://gitlab.kitware.com/ben.boeckel/cmb-lfs-test.git > real 0m18.448s > > This is with git 2.13.3 > Definitely faster, and easier to follow output, but both pretty fast from 21 on a desktop Linux machine. I am pretty sure it has a lot to do with your proximity to the data. At KRS, it took git clone, Ben's test repo 21m32s git lfs clone, Ben's test repo 6m36s git clone, CMB test-data repo 2m28s This is git 2.11.0 on macos Sierra. I also installed homebrew git (2.13.3) and got a similar time for "git lfs clone" (6m49s). We are planning to move ahead with git-lfs but will prune the larger, unused input datasets by not adding them to the repo. If we need large data (examples or benchmarking) a separate repo is an option. David From marcus.hanwell at kitware.com Tue Jul 25 15:38:40 2017 From: marcus.hanwell at kitware.com (Marcus D. Hanwell) Date: Tue, 25 Jul 2017 15:38:40 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> <20170725155404.GA485@megas.kitware.com> Message-ID: On Tue, Jul 25, 2017 at 3:32 PM, David Thompson wrote: > Hi Marcus, > > > That is odd, I was interested, and just tried cloning > > > > $ time git clone https://gitlab.kitware.com/ben.boeckel/cmb-lfs-test.git > > real 0m39.754s > > $ time git lfs clone https://gitlab.kitware.com/ > ben.boeckel/cmb-lfs-test.git > > real 0m18.448s > > > > This is with git 2.13.3 > > Definitely faster, and easier to follow output, but both pretty fast > from 21 on a desktop Linux machine. > > I am pretty sure it has a lot to do with your proximity to the data. At > KRS, it took > Totally agree, just trying to add some data points. It would be good to compare with GitHub too as we would both be remote - at best you can saturate KHQ's 150Mbps connection, but it could be far worse that that too. > > git clone, Ben's test repo 21m32s > git lfs clone, Ben's test repo 6m36s > git clone, CMB test-data repo 2m28s > > This is git 2.11.0 on macos Sierra. I also installed homebrew git (2.13.3) > and got a similar time for "git lfs clone" (6m49s). > If I understand correctly, Ben was saying newer git makes git clone perform nearly as well as git lfs clone. > > We are planning to move ahead with git-lfs but will prune the larger, > unused input datasets by not adding them to the repo. If we need large data > (examples or benchmarking) a separate repo is an option. Sounds good, we will probably use this approach for Tomviz, but likely on GitHub with their 1GB limit on LFS data size. We haven't gotten to it yet, so certainly interested in new findings for an already quite large data repository. Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.boeckel at kitware.com Tue Jul 25 15:57:49 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Tue, 25 Jul 2017 15:57:49 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> <20170725155404.GA485@megas.kitware.com> Message-ID: <20170725195749.GA8987@megas.kitware.com> On Tue, Jul 25, 2017 at 15:38:40 -0400, Marcus D. Hanwell wrote: > Sounds good, we will probably use this approach for Tomviz, but likely on > GitHub with their 1GB limit on LFS data size. We haven't gotten to it yet, > so certainly interested in new findings for an already quite large data > repository. Note that it sounds like there are also monthly bandwidth limits on Github LFS repos. Dan LaManna says there's a repo on Github he's working on that can't be cloned because its LFS quota has been reached. I don't know what that limit is though. We also need to figure out LFS data mirroring between our repos and Github (both directions) at some point for the repos that need it. --Ben From ben.boeckel at kitware.com Tue Jul 25 16:14:58 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Tue, 25 Jul 2017 16:14:58 -0400 Subject: [Smtk-developers] LFS configuration options Message-ID: <20170725201458.GA25899@megas.kitware.com> Hi, So looking at LFS configuration knobs, these seem interesting: * `lfs.storage` Allow override LFS storage directory. Non-absolute path is relativized to inside of Git repository directory (usually `.git`). Note: you should not run `git lfs prune` if you have different repositories sharing the same storage directory. Default: `lfs` in Git repository directory (usually `.git/lfs`). This should make it possible to share LFS directories. It is a per-clone option though. Or you could have one global LFS data store I suppose. * `lfs.fetchinclude` When fetching, only download objects which match any entry on this comma-separated list of paths/filenames. Wildcard matching is as per git-ignore(1). See git-lfs-fetch(1) for examples. * `lfs.fetchexclude` When fetching, do not download objects which match any item on this comma-separated list of paths/filenames. Wildcard matching is as per git-ignore(1). See git-lfs-fetch(1) for examples. If you'd like to ignore specific files in a clone. --Ben From marcus.hanwell at kitware.com Wed Jul 26 10:58:14 2017 From: marcus.hanwell at kitware.com (Marcus D. Hanwell) Date: Wed, 26 Jul 2017 10:58:14 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: <20170725195749.GA8987@megas.kitware.com> References: <20170721202846.GA11628@rotor.kitware.com> <9F8CD344-7F34-4AA4-A33B-277D98E45C53@kitware.com> <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> <20170725155404.GA485@megas.kitware.com> <20170725195749.GA8987@megas.kitware.com> Message-ID: On Tue, Jul 25, 2017 at 3:57 PM, Ben Boeckel wrote: > On Tue, Jul 25, 2017 at 15:38:40 -0400, Marcus D. Hanwell wrote: > > Sounds good, we will probably use this approach for Tomviz, but likely on > > GitHub with their 1GB limit on LFS data size. We haven't gotten to it > yet, > > so certainly interested in new findings for an already quite large data > > repository. > > Note that it sounds like there are also monthly bandwidth limits on > Github LFS repos. Dan LaManna says there's a repo on Github he's working > on that can't be cloned because its LFS quota has been reached. I don't > know what that limit is though. > It is 1GB of bandwidth too, pretty low, then at $5 I think you get a data pack that gives you 50GB. > > We also need to figure out LFS data mirroring between our repos and > Github (both directions) at some point for the repos that need it. > > Certainly, for those that want to do it, it seemed pretty easy in my tests from a local perspective - pushing to our Gitlab versus GitHub looked identical. Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.boeckel at kitware.com Wed Jul 26 11:19:34 2017 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Wed, 26 Jul 2017 11:19:34 -0400 Subject: [Smtk-developers] git-lfs test repository In-Reply-To: References: <20170724145854.GD23225@megas.kitware.com> <20170724155005.GA2206@rotor.kitware.com> <20170725155404.GA485@megas.kitware.com> <20170725195749.GA8987@megas.kitware.com> Message-ID: <20170726151934.GB6007@megas.kitware.com> On Wed, Jul 26, 2017 at 10:58:14 -0400, Marcus D. Hanwell wrote: > On Tue, Jul 25, 2017 at 3:57 PM, Ben Boeckel > wrote: > > We also need to figure out LFS data mirroring between our repos and > > Github (both directions) at some point for the repos that need it. > > > Certainly, for those that want to do it, it seemed pretty easy in my tests > from a local perspective - pushing to our Gitlab versus GitHub looked > identical. Yeah, but we use some low-level git process for it (bare repos, refspecs, and other fun). I've opened an issue on the project which hosts the code for it to add LFS support, but just a note that it isn't there (yet). --Ben